Jev-Mobile: Jev as an Executor for Mobile GUI Agents

{'Problem': 'Existing systems for mobile GUI agents typically rely on Vision-Language Models (VLM) for both planning and action grounding, which results in increased latency and higher model-serving costs. This paper addresses these inefficiencies by proposing a new architecture that separates planning and execution processes, thereby enhancing performance in mobile environments. The work is presented as a preprint and has not yet undergone peer review.', 'Method': 'The proposed architecture, Jev-Mobile, employs a low-frequency VLM for planning while utilizing a high-frequency lightweight execution mechanism. The action space is defined through an accessibility tree, which structures the executable actions available to the agent. The decision-making process is governed by Jev, a fast typed decision model that allows for rapid action selection within the defined action space. This dual approach aims to optimize the execution of tasks in mobile GUI environments by minimizing the overhead associated with traditional VLMs.', 'Results': 'Jev-Mobile achieves a task success rate of 79%, which is competitive compared to SeeAct-V at 78% and Step-wise VLM at 84%. Additionally, it demonstrates a mean end-to-end execution time reduction of 32.7% when compared to Step-wise VLM, indicating significant efficiency improvements. Furthermore, Jev-Mobile reduces the mean model API cost by 73.4% relative to Step-wise VLM, showcasing its potential for cost-effective deployment in mobile applications.', 'Limitations': 'The authors do not report any limitations in the study. However, the lack of peer review may imply that the findings should be interpreted with caution until validated by the community.', 'Why it matters': 'The implications of this work are significant for the development of mobile GUI agents, as it presents a framework that can reduce both latency and operational costs. By decoupling planning from execution, future research can explore further optimizations and adaptations of the Jev-Mobile architecture in various application domains, potentially leading to more responsive and efficient mobile interfaces.'}