Abstract: In recent years,Visual Transformer (ViT) models have been widely deployed in edge-based visual applications because of their powerful image understanding capabilities.To achieve optimal inference accuracy-latency balance in resource-constrained edge-side inference,it is essential to scale ViT models effectively based on available resources.However,existing inference model scaling techniques can only perform scaling at the entire model granularity,leading to the loss of critical information and often requiring more computational resources or higher inference latency to achieve equivalent accuracy.This paper proposes LegoViT,a method that identifies scalable model blocks from the feedforward networks of ViT mo-dels,thus supporting runtime block-level model scaling.Comparative test results demonstrate that LegoViT achieves a 22.37% reduction in memory footprint of ViT models,a 21.1% decrease in computational overhead,and an average 61.05% reduction in inference latency.

Key words: Edge side, ViT, Inference optimization, Block-grained scaling