GeoTP: parameter-level intelligent planner of GIS tool-use chains based on instruction tuning of large language models

Geographic Information Systems (GIS), a core tool for spatial analysis and decision support, have been widely applied in various fields, including urban planning^[1,2], traffic management^[3], disaster response^[4,5], and resource management^[6]. Faced with increasingly complex spatial analysis needs, it is crucial to precisely configure each tool's parameters (e.g., buffer distance, field expression) to solve problems. Automating this process by generating executable workflows with correct tool sequences and compliant parameter settings directly from natural language instructions is crucial for significantly improving the efficiency of GIS applications and lowering the barrier to spatial analysis^[7,8].

Rule-based systems (e.g., ArcGIS ModelBuilder) are classic solutions for formalizing specific workflows^[9,10], but they are inflexible and difficult to adapt to dynamic changes in tasks^[11]. In recent years, large language models (LLMs) have demonstrated powerful capabilities in semantic understanding, reasoning, and code generation, providing new approaches to solving intelligent geospatial tasks^[12,13]. Researchers have begun to explore the deep integration of LLMs and GIS^[14,15], forming a range of parallel research methods and technical approaches, aiming to achieve automation at different levels and in different scenarios.

Deeply integrating LLMs into existing GIS platforms (e.g., QGIS) has become the mainstream practice for achieving copilot-style interaction. Such research aims to overcome the challenges of integrating AI into complex GIS environments, including tool diversity, heterogeneity in data standards, and the accurate conversion of natural language into GIS commands^[16]. The GIS Copilot^[17] plugin guides LLMs in generating accurate PyQGIS code through a structured library of tool descriptions and a code review module, demonstrating a high success rate for basic and intermediate tasks. This type of work marks the evolution from externally invoked tools to in-platform intelligent agents. Its advantage lies in the close integration of the GIS software's computing ecosystem and visualization capabilities, which enhances analytical transparency and control over results. Utilizing the planning and tool invocation capabilities of commercial LLMs has become a key pathway to achieve high-performance, general-purpose automated systems^[18]. The GeoGPT framework centers on GPT-4, which autonomously plans and invokes a predefined toolset through prompt engineering to complete the entire workflow from data collection to visualization^[19]. Such systems demonstrate exceptional accuracy when handling complex workflows involving up to eight steps, highlighting the pivotal role of powerful LLMs in complex task planning.

However, the outstanding performance of LLMs largely relies on closed-source commercial models such as GPT-4, which present significant limitations regarding data privacy, localized deployment, and long-term usage costs. Additionally, LLM-based approaches still face challenges, including tool semantic ambiguity and output hallucinations^[20]. To address LLMs' reliance on domain-specific knowledge and their hallucination issues, retrieval-augmented generation (RAG) technology^[21] has been systematically integrated into geospatial task automation frameworks. RAG provides real-time, reliable knowledge support for LLMs' decision-making and generation processes by retrieving relevant information from external knowledge bases, such as authoritative scientific reports, professional literature, or structured tool documentation, thereby significantly enhancing the accuracy and reliability of the output^[22]. In the field of GIS, RAG is regarded as a potential solution to address the issue of lengthy tool documentation that exceeds the contextual limits of models, enabling LLMs to dynamically retrieve parameter details and usage examples for required tools. GeoCogent systematically constructs a multimodal geospatial knowledge base and deeply integrates the RAG mechanism, effectively improving the accuracy of code generation^[23]. However, its performance remains constrained by retrieval quality and real-time responsiveness, and it cannot fundamentally internalize domain knowledge as a stable capability of the model. To fundamentally reduce reliance on closed-source models and develop an open-source LLM capable of private deployment and intelligent, parameter-level planning within GIS tool-use chains, researchers have shifted toward deeply empowering open-source models through instruction tuning^[24]. Parameter-efficient fine-tuning techniques, such as LoRA, are favored for their ability to incorporate domain knowledge efficiently. Zhang et al. applied LoRA to fine-tune LLaMA-2-7B, training a specialized model capable of generating GIS tool-use chains^[25]. Chen et al. significantly improved the understanding and generative performance of open-source models in the geoscience domain through a combined strategy of continued pre-training and instruction tuning^[26]. These efforts preliminarily validate that, through domain-specific data fine-tuning, open-source LLMs can acquire capabilities to handle geospatial tasks. Researchers from GeoGPT and GeoCogent have also clearly indicated that developing specialized local LLMs tailored for the GIS field is a crucial future direction to reduce reliance on commercial APIs and enhance domain adaptability.

Despite significant progress, current solutions based on open-source LLMs still face several bottlenecks. First, their planning accuracy and parameter compliance in complex tasks remain significantly weaker than those of closed-source models such as GPT-4, which limits their practical deployment. Second, existing datasets largely rely on manual construction, resulting in small scale and low diversity, and particularly lack standardized constraints on tool parameters. Consequently, the models tend to generate syntactically correct but semantically erroneous parameter configurations. Third, there is a lack of systematic evaluation benchmarks to comprehensively assess model performance in tool-use chain planning and parameter configuration. To address these challenges, this study proposes an instruction tuning training framework for open-source LLMs for geospatial tasks. The core contributions are as follows:

(1) A structured method for geospatial tool description and parameterized instruction data generation.By introducing a self-instruct generation strategy and incorporating domain knowledge, we guide a large language model to automatically generate diverse, multi-complexity 'instruction–tool chain' pairs. This approach places particular emphasis on the structured representation of parameter templates, data types, and value constraints, thereby laying a foundation for the model to learn precise tool invocation logic and parameter adaptation.

(2) A high-quality instruction tuning dataset, GeoITC, and a systematic evaluation benchmark, GeoITC-Eval. The dataset covers single-tool, dual-tool, and multi-tool tasks, and extends to unseen tools and extra-long workflow scenarios, forming a multi-level evaluation system. Model performance is comprehensively quantified from two dimensions: tool chain order accuracy (CTOA) and parameter accuracy (PA).

(3) A high-performance geospatial tool orchestration model, GeoTool Planner (GeoTP).Based on the LLaMA-3-8B model, we inject geospatial tool usage capabilities via LoRA. Experimental results show that GeoTP achieves outstanding performance across multiple tasks. It not only substantially outperforms comparable models on known tool tasks, but also demonstrates robust generalization to unseen tools and long-chain workflow planning.

The framework of the proposed method, illustrated in Fig. 1, consists of three components: dataset construction, instruction tuning, and GIS tool planning.

Figure 1. 

The framework of the GIS tool planning model GeoTP.

2.1.1.   Construction of the instruction tuning dataset

Two datasets, GeoITC and the evaluation dataset GeoITC-Eval, were constructed for fine-tuning General LLMs to enhance their capability in solving Geospatial Tasks. To overcome the bottlenecks of scarce geospatial instructional data and high manual design costs, a data-efficient construction approach that combines the self-guidance generation strategy with a Simulated Environment was proposed. First, the collected GIS tools are standardized and presented with parameterized descriptions to establish a standard format that includes tool functions, structured parameter templates, data constraints, and examples. Based on this, the high-performance LLMs are guided to automatically generate diverse geospatial task instructions of varying levels of complexity (single-tool, dual-tool, and triple-tool) within a simulated GIS execution environment, while simultaneously producing their corresponding, fully parameterized tool-use chain solutions.

2.1.2.   LLM instruction tuning

Based on the open-source general-purpose LLM LLaMA-3-8B, the model is fine-tuned using the LoRA Fine-tuning technique. This approach freezes the original weights of the base model (LLaMA-3-8B) and introduces trainable Low-rank matrices alongside key modules such as attention for adaptive adjustment. It significantly reduces the number of training parameters, thereby lowering GPU memory consumption and training time. By applying LoRA fine-tuning on the high-quality dataset GeoITC, the model learns to map natural language descriptions to structured tool-use chains and parameter configurations. The resulting GIS tool intelligent planning model, GeoTP, retains the general language capabilities of the Base model while accurately mastering the planning and execution of complex geospatial workflows.

2.1.3.   GIS tool planning

Driven by natural-language task instructions, the fine-tuned LLMs autonomously orchestrate GIS tool-use chains and configure parameters. When entering an instruction, the primary objectives of the geospatial analysis task and the corresponding input data requirements must be clearly defined, with explicit annotations of data types and storage paths. This ensures that the task instruction is complete and semantically accurate, without requiring the specification of GIS tools, operational steps, or parameter settings.

2.2.   Construction of the instruction tuning dataset

The core objective of instruction tuning is to align the model's output with the dialogue style and knowledge system of human experts or specific domains, thereby significantly enhancing the model's ability to understand and execute complex tasks based on given instructions. To achieve this goal, constructing a high-quality instruction tuning dataset is crucial, and obtaining accurate 'instruction–response' pairs is particularly important. This process requires not only that the tool-use chain logic in the responses be correct and comprehensive, but also that the parameter configuration of each tool strictly comply with the standards and semantic constraints of the geospatial domain. For example, when the model is tasked with identifying suitable areas for park construction in urban planning—ensuring that the area is within 500 m of a school and avoids water protection zones—it must accurately parse multiple geographic constraints and plan the correct sequence of spatial analysis steps. These steps include selecting appropriate tools and setting corresponding parameters, such as the buffer distance in 'buffer analysis' and the layer processing method in 'overlay analysis', ultimately generating a complete and executable tool-use chain. The ability to understand and configure tool functions and parameters is essential to enabling the model to convert users' natural-language descriptions into a professional and reliable sequence of operations. It also serves as the foundation for deploying practical LLMs in specialized domains.

Currently, the self-instruct strategy is a widely adopted and efficient approach for constructing Instruction Tuning Datasets. This strategy guides LLMs themselves to participate in data generation, automatically creating 'instruction–solution' pairs and thereby reducing reliance on manual annotation. The process consists of two stages: instruction generation and solution construction, with the latter focusing on producing executable sequences of operations that include specific tool invocations and parameter configurations. In this study, the strategy is employed to automatically generate supervised data using a high-performance LLM, covering diverse geospatial tasks, their corresponding tool-use chains, and parameterized operations. This provides structured and large-scale training resources that enable precise planning and execution of the model in complex geospatial scenarios. The geospatial domain-specific model presented in this study is built on a general LLM and further refined through instruction tuning in the geospatial domain. This process enables the model to acquire professional capabilities such as understanding spatial analysis tasks, autonomously orchestrating tool-use chains, and appropriately configuring parameters.

Because DeepSeek-V3.1 performs exceptionally well in structured output and tool-calling tasks and can stably generate syntactically correct and logically coherent tool chains, the model has implicitly absorbed some GIS tool knowledge during large-scale pretraining, making it better aligned with the needs of this study for geographic information scenarios. Moreover, the model supports a context window of up to 128 K tokens, which enables it to effectively handle long task descriptions involving multi-tool nesting and complex parameter constraints^[27]. Therefore, this paper selects it as a high-performance teacher model and uses structured prompt engineering to explicitly express its implicit spatial inference knowledge, thereby generating the high-quality dataset GeoITC to empower small open-source models that can be deployed locally.

2.2.1.   Structured description of QGIS tools

This study aims to guide open-source LLMs in developing a deeper understanding of the functional characteristics, parameter settings, and applicable contexts of GIS tools, with a particular focus on enabling them to master multi-tool collaborative methods for specific geospatial tasks. Relevant tool documentation was systematically collected, and textual descriptions of their functions and parameters were extracted. Specifically, the original documentation of 18 commonly used QGIS tools was standardized, and a unified structured description template was designed for each tool, as shown in Eq. (1). Where $ {n}_{i} $ represents the tool name, $ {f}_{i} $ indicates the tool function description, $ {p}_{i} $ denotes the parameter design template, $ {r}_{i} $ specifies the input data requirements, $ {e}_{i} $ provides an example of using the tool, and $ T $ stands for the complete tool combination.

Table 1 presents the names, functions, and parameters of the three tools, with the complete tool list provided in Supplementary Table S1. The structured description template for the tools is shown in Fig. 2 and primarily includes algorithm functions, parameter explanations, and data constraints. Although in actual engineering deployment the final storage path can be fully distributed automatically by the underlying GIS platform, this study still treats it as a mandatory generated parameter during the experimental evaluation stage. The purpose of this design is to maximize the testing of the large model's instruction-following capability under strong constraints and its alignment accuracy with a specific location. This structured format helps reduce the complexity of the input text and alleviates the understanding burden on the LLMs.

Figure 2. 

Structured tool description template.

2.2.2.   Geospatial task generation

To construct diverse and parameter-complete descriptions of geospatial tasks, this study employs the collected QGIS toolset and its structured description template, applying a Self-Instruct strategy^[28] to guide a general LLM in generating geospatial task descriptions that encompass a wide range of scenarios with well-defined parameter settings. Given the strong semantic understanding and planning capabilities of DeepSeek-V3.1, this study primarily relies on it to implement geospatial task generation using the self-instruct strategy.

In practice, given the diversity and complexity of geospatial tasks, three distinct prompt templates were designed for single-tool, dual-tool, and triple-tool tasks. In the single-tool task generation template (Fig. 3), the model is explicitly required to generate instructions that depend solely on the specified tool and include complete parameter settings, covering both the data path and parameter values. To enhance the model's understanding of the correspondence between tasks and parameters, manually constructed example instructions containing parameterized samples were incorporated into the prompt. In the dual-tool (Fig. 4) and triple-tool (Fig. 5) task-generation templates, the structure is similar but emphasizes integrating the tools into a coherent workflow, with parameters between successive steps properly linked and transferable. Through this hierarchical prompting system, geospatial task data spanning diverse scenarios and complexities were successfully collected, providing high-quality supervision signals with rich parameters and rigorous logic for model training.

Figure 3. 

The prompt for single-tool geospatial task generation.

Figure 4. 

The prompt for dual-tool geospatial task generation.

Figure 5. 

The prompt for three-tools geospatial task generation.

This approach not only alleviates the scarcity of geospatial task instruction data but also ensures the accuracy and executability of generated tasks in tool invocation and parameter configuration through structured parameter descriptions and hierarchical prompt design. It provides a reliable means to enhance the LLMs' capability in planning and parameterized tool use within professional GIS fields.

2.2.3.   Parameterized instruction tuning data generation

For handling geospatial tasks, the core objective lies in generating a complete solution with a correct sequence of tools and accurate parameter configuration. Traditionally, obtaining such a solution relied on manual operations and process documentation by GIS experts, a method that is inefficient and difficult to scale up. To address this, this study adopts an automatic generation strategy based on an LLM. The core lies in guiding the model, through carefully designed prompt engineering, to produce parameterized and executable workflows. The specific prompt design is illustrated in Fig. 6. First, a structured description of tools is provided to the model, enabling it to accurately understand the tool call interfaces and thereby establish a foundation for generating reasonable parameter values. Second, after understanding the task objective, the model needs to plan the tool execution sequence and perform the critical task of parameter adaptation. This requires the model to parse the task context (for example, 'Calculate the road length of main roads along both banks of the Yangtze River in Wuhan city, with the input data being the regional boundary file wuhan_river_area.shp') and recognize that the input file wuhan_river_area.shp defines the spatial scope of the task. The model then infers that obtaining road data within this boundary is the primary step. It understands that 'both banks of the Yangtze River' represents a filter condition based on a spatial relationship, but also realizes that data directly downloaded from an open-source map such as OSM typically do not contain this attribute. Therefore, it plans to first retrieve all road data, followed by spatial analysis for filtering if necessary. This reasoning guides the model to initially select the OSM Downloader Tool and correctly map the spatial extent of wuhan_river_area.shp to the Download Area parameter of that tool. The model recognizes that 'calculating length' requires precise geometric computation, while the downloaded geographic data are typically in a Geographic Coordinate System. Therefore, a projection transformation must be performed to obtain lengths measured in meters. This leads the model to sequentially include the Reproject Layer and Field Calculator tools in the tool-use chain. It can automatically set the output of the previous step as the input for the next, forming a coherent data flow. More importantly, it can generate the Professional Expression $length for the Field Calculator tool to compute the Geometric Length of each line feature after projection, and name the resulting field road_length_m, demonstrating an accurate understanding of GIS professional operations and semantics. It should be noted that although this paper represents tool combinations in sequential form, this design is capable of expressing complex multi-branch topologies such as tree structures and Directed-Acyclic Graphs (DAGs). In the design of the GeoITC dataset, dependencies between tools are not limited to adjacent upstream and downstream relationships; subsequent tools can explicitly reference parameter variables to accept the outputs of multiple preceding tools as inputs simultaneously.

Figure 6. 

The prompt used for parameterized tool-use chain generation.

The specific parameter values in this dataset are strictly treated as learning examples, rather than fixed rules that the model is expected to memorize. Our design objective is to teach the model, through a large number of examples covering different numerical values and data sources, to understand the semantic typing of parameters, their numerical constraints, and their contextual roles in chained propagation. Whether the model can dynamically generate adaptive parameters based on new task descriptions and data files will be evaluated by its generalization performance on unseen tools and long-chain tasks. This design aims to cultivate the model's ability to abstract general parameter inference capabilities from concrete examples, rather than template matching.

Since LLMs cannot directly execute GIS operations, a structured output format is designed to simulate this process. Specifically, the model's output must include two parts: first, an assessment of task feasibility; if the task is solvable, the model should then generate a high-level solution framework and produce a workflow that includes a detailed sequence of tools, parameter configuration, and data flow. To enhance the reliability of the generated results, this design incorporates a self-verification mechanism: the sequence and functional intent of the tools planned in the framework must remain consistent with the specific invocations and parameter settings in each step of the workflow. Any inconsistency indicates the presence of hallucinations or logical errors in the output and should therefore be discarded. Studies have shown that this step-by-step inference approach of 'planning first, then refining'^[29] can effectively enhance the accuracy and reliability of model responses, thereby enabling the systematic acquisition of high-quality and verifiable geospatial task tool-use chains.

The GeoITC constructed in this study is a parameterized tool-use chain Instruction Tuning Dataset for geospatial tasks, built based on the self-guidance generation strategy (Eq.[2]). The term instruction denotes the instruction task, and output represents the parameterized tool-use chain. To generate Instruction Tuning data with varying levels of complexity, this study collected three levels of geospatial tasks along with their corresponding tool-use chains, including single-tool, dual-tool, and triple-tool tasks. Specifically, for single-tool tasks, each tool was repeated 35 times to help LLMs fully understand its functions and usage methods (18 × 35 = 630). All dual-tool combinations were repeated 20 times to enable LLMs to learn how different tool combinations can be used to solve geospatial tasks (18 × 17 ÷ 2 × 20 = 3,060). After completing the alignment between the framework and the workflow, unmatched cases were removed, resulting in a total of 2,740 dual-tool instructions. For the triple-tool tasks, all triple-tool combinations were repeated four times, generating 18 × 17 × 16 ÷ 3 ÷ 2 × 4 = 3,264 instructions. After excluding unmatched cases, 2,088 instructions were retained. Detailed statistics of all collected instruction tuning data are presented in Table 2, where the tool numbers in Table 2 indicate the number of tools required to solve the corresponding geospatial tasks.

2.2.4.   GeoITC-Eval benchmark construction

To assess the capability of language models in solving geospatial tasks, the evaluation dataset GeoITC-Eval was collected. Similar to the instruction tuning data, three levels of Geospatial Tasks were gathered. For tasks that require only a single tool, a total of 270 tasks were collected (18 tools, each repeated 15 times). For tasks requiring two tools, the 18 available tools were paired in all possible combinations to form nine unique pairs; this process was repeated ten times, resulting in 90 tasks. For tasks requiring three tools, the 18 tools were grouped into sets of three to form six unique triples; this process was also repeated ten times, yielding 60 tasks in total. Unlike the instruction tuning data collected solely through Deepseek-V3.1, the evaluation data require professional verification. Specifically, the details and framework of each geospatial task, as well as every step of the workflow, were reviewed by two GIS experts holding master's degrees. This process ensures that the evaluation data can be effectively used to assess model performance. Table 3 presents the statistical results of all collected evaluation data.

In addition, two extended datasets were collected to evaluate the performance of GeoTP in scenarios beyond the training data. Specifically, the first extended tool-use chain length dataset, GeoITC-EvalProlong, includes complex tasks that require four to five GIS tools to complete. Using GeoITC-EvalProlong, GeoTP can be tested on tasks that are more complex than those in the training data. Furthermore, three external GIS tools were collected, and the second extended tool dataset, GeoITC-EvalExtend, was constructed based on tools that were not used during GeoITC training. Table 4 presents information on these three tools. Specifically, three levels of geospatial tasks were collected: for geospatial tasks requiring a single tool, 15 instructions were gathered (five repetitions for each of the three tools); for geospatial tasks requiring dual-tool collaboration, 15 instructions were collected, each based on one external tool and one of the 18 training tools. This study collected 15 instructions for geospatial tasks that require three tools to complete. Each instruction is based on one external tool and two of the 18 training tools. In this way, GeoITC-EvalExtend can be used to evaluate the performance of GeoTP when operating with unfamiliar GIS tools, thereby demonstrating its transfer capability.

2.3.   Efficient parameter fine-tuning based on LoRA

Given the limited computational capacity of local devices, full-parameter fine-tuning of LLMs is not feasible. Therefore, a parameter-efficient fine-tuning strategy must be adopted to make full use of local computational resources while optimizing the model and minimizing computational and storage demands during training. In this study, the LoRA method is employed within the GeoTP framework due to its outstanding performance demonstrated in recent research on Instruction tuning. Compared with full-parameter fine-tuning of LLMs, LoRA can significantly reduce the number of trainable parameters. For the weight matrix of a large pre-trained language model, the low-rank adaptation (LoRA) method freezes the matrix and constrains its update $ \Delta W $ through a low-rank decomposition, where $ B\in {\mathbb{R}}^{d\times r} $ and $ A\in {\mathbb{R}}^{r\times k} $ are two trainable parameters, and the rank is $ r\ll \min (d,k) $. For the linear layer $ h={W}_{0}x $, its modified forward propagation is expressed as $ h $ (Eq. [3]). It is worth noting that the adopted LoRA method is scalable and can adapt to different system resources. When computational resources are significantly limited, the LoRA configuration can be adjusted by reducing the number of elements in matrix, which are then decomposed into low-rank matrices A and B. In addition, the ranks of matrices $ A $ and $ B $ can also be reduced.

To comprehensively quantify the quality of workflows generated by LLMs, this study evaluates them from two dimensions: tool-use chain accuracy and parameter accuracy. Tool-use chain accuracy (Tool Correctness and Order) assesses whether the model can plan the use of all necessary tools and whether the sequence of tool invocations is reasonable. This dimension focuses on the overall structure and logical flow of the workflow, requiring the generated workflow to be fully consistent with the expected workflow in terms of the number of tools, identifiers, and their order. Parameter accuracy verifies whether the parameters configured by the model for each tool correctly match the tool definitions, including the presence of all required parameters, conformity of parameter value data types, and compliance of value ranges. In the quantitative evaluation, the tool correctness and order accuracy (TCOA) is used to calculate the proportion of tool-use chain samples with correct tool usage order to the total number of samples (Eq. [4]). Here, $ {N}_{s} $ denotes the total number of samples, $ {N}_{t} $ represents the number of samples with a completely correct tool-use chain, and I[·] is the indicator function, which equals 1 when the condition inside the parentheses is satisfied and 0 otherwise. The symbol $\land $ denotes a logical AND operator, requiring that both the preceding and following conditions be satisfied simultaneously. The criterion for a completely correct tool-use chain $ Correc{t}_{toolchain} $ must meet the following two conditions: the number of tools $ m $ in the generated tool-use chain $ G $ must be exactly the same as that in the expected tool-use chain $ E $, and the tool names at each corresponding position must fully match those in the expected tool-use chain.

The calculation of Parameter Accuracy (PA) is shown in Eq. (5), which is used to determine the proportion of tool-use chain samples with fully reasonable parameter settings to the total number of samples. Here, $ {N}_{s} $ denotes the total number of samples, and $ {N}_{p} $ represents the number of tool-use chain samples with fully reasonable parameter settings. Similarly, I[·] is the indicator function, and $ {T}_{j} $ denotes the j-th generated tool-use chain. For each tool within the tool-use chains, the parameter validity criterion $ Correc{t}_{params} $ must simultaneously satisfy the following three conditions: the parameter names $ PG $ in the generated tool-use chains must exactly match the parameter names $ PT $ in the toolset; the data type of each parameter value must comply with the tool's definition requirements; and the parameter value must fall within the tool's specified range of parameter values.

3.2.   Baseline model and comparison strategy

Selecting appropriate LLMs is essential for elucidating how instruction tuning enhances the capability of LLMs in geospatial tasks. LLaMA-3-8B has highly robust general-purpose semantic comprehension and logical mapping capabilities, and its open-source ecosystem and tool-chain are the most complete, enabling it to serve as a standard capability anchor for validating the universality of the proposed method frame. The core API documentation and community code for QGIS and most advanced open-source GIS toolchains are primarily in English, and the extremely high representational density of the LLaMA series models on English corpora helps the model quickly internalize structured QGIS tool descriptions during fine-tuning. With 8 billion parameters, it is currently the optimal size for efficient private fine-tuning via LoRA on consumer-grade GPUs, allowing it to maintain extremely high inference speed while ensuring sufficient capacity for domain knowledge. Therefore, this paper adopts the LLaMA-3-8B model. In addition, to compare the performance of different LLMs, DeepSeek-V3.1^[30] and Qwen3-max^[31] were evaluated on the collected evaluation dataset. This comparison helps determine whether, after applying LoRA fine-tuning using the collected instruction tuning dataset to smaller open-source models, their performance can surpass that of the most advanced open-source LLMs currently available.

Since all available QGIS tool information has been integrated into the GeoTP model through the collected instruction tuning dataset, GeoITC during the LoRA fine-tuning process, the evaluation only requires inputting the geospatial task into the GeoTP model. However, to ensure an effective and fair comparison, a strategy suitable for general-domain baseline LLMs was designed. Specifically, for the Deepseek-V3.1 and Qwen3-max models, a system prompt was created to provide them with the collected GIS tools and their corresponding descriptions, as shown in Fig. 7.

Figure 7. 

Baseline models generate prompts for parameterized tool-use chains.

3.3.   Experimental results and analysis

3.3.1.   Performance evaluation based on the standard evaluation set (GeoITC-Eval)

Performance Evaluation Based on the Standard Benchmark (GeoITC-Eval) Table 5 presents the performance comparison between the baseline models and GeoTP on the GeoITC-Eval evaluation dataset. To assess the models' capabilities under varying task complexities, the number of tools required for each task type (ranging from one to three) was recorded. Overall, GeoTP, trained with instruction tuning, significantly outperforms other baseline models in most cases, particularly in tool-use chain accuracy. Specifically, in single-tool tasks, GeoTP achieves a TCOA of 91.5%, far exceeding the baseline models, indicating that it has approached expert-level performance in basic tool invocation and sequential planning. This result further shows that relying solely on prompt engineering makes it difficult for general-purpose large language models to acquire geospatial analysis logic understanding and parameterized decision-making capabilities comparable to those enabled by domain-specific Instruction Tuning. Its parameter accuracy (PA) also reaches 90.0%, demonstrating the model's ability to precisely match parameter types and values, which further validates the effectiveness of parameterized tool descriptions in the instruction data. In multi-tool tasks (involving two to three tools), as task complexity increases, the performance of all models declines to some extent; however, GeoTP still maintains a leading position. In particular, in tool-use chain planning (TCOA: 65.6% for two tools and 58.3% for three tools), it significantly outperforms models of comparable scale such as LLaMA-8B and the general-purpose model Qwen3-max. This indicates that training with structured tool-use chains data enables the model to perform multi-step reasoning and tool integration more effectively. It is noteworthy that although DeepSeek-V3.1 achieves slightly higher parameter accuracy in certain tasks (e.g., three-tool PA) than GeoTP, its tool-use chains accuracy remains consistently lower.

This further demonstrates that instruction tuning data specifically designed for geospatial tasks plays a crucial role in enhancing the model's capability in professional tool utilization. In summary, GeoTP's outstanding performance in both tool-use chains planning and parameter configuration validates the effectiveness of the parameterized and structured instruction tuning dataset (GeoITC) proposed in this study for training domain-specific models. It significantly enhances the ability of open-source models to perform tool invocation and workflow generation in complex geospatial tasks.

3.3.2.   Generalization capability evaluation based on the long tool-use chains task dataset (GeoITC-EvalProlong)

In more complex task scenarios (with four and five tools), the performance of GeoTP and the baseline models was further evaluated on the GeoITC-EvalProlong dataset. It should be noted that during training, GeoTP was exposed only to tasks involving up to three tools; therefore, such tasks impose higher demands on its generalization and compositional reasoning capabilities. As shown in Table 6, GeoTP still maintains a significant advantage over the general baseline models in tool-use chain planning and parameter configuration (PA). This indicates that fine-tuning the model with the structured and parameterized instruction data (GeoITC) proposed in this study enables it to generalize the learned tool combination logic to longer task chains. It is noteworthy that although GeoTP demonstrated excellent parameter accuracy (PA: 76.7%) in the four-tool tasks, all performance metrics declined substantially when the tool-use chain was extended to five tools. This indicates that the current approach still requires improvement in planning and parameter configuration stability when dealing with highly complex workflows. Nevertheless, GeoTP's relatively superior performance, particularly its ability to maintain high parameter configuration consistency in the four-tool tasks, provides preliminary evidence that the proposed data construction method helps the model sustain better logical coherence in complex scenarios. This offers a valuable direction for developing specialized models capable of handling long-chain geospatial tasks.

3.3.3.   Transferability assessment based on external tool task dataset (GeoITC-EvalExtend)

Finally, to examine the model's generalization and transferability when encountering external tools unseen during training, GeoTP and the baseline models were evaluated on the GeoITC-EvalExtend dataset, as shown in Table 7. All tasks in this dataset were constructed using tools that were not employed during model training. The experimental results indicate that GeoTP outperformed all baseline models across both dimensions of tool-use chain accuracy and parameter accuracy (PA). In particular, for tasks requiring two or three tools, GeoTP achieved a CTOA of 53.3% in both cases, which is significantly higher than that of the second-best model, Qwen3-max (40.0%). This demonstrates that, after being trained with the structured and parameterized instruction data designed in this study, the model can not only correctly plan the execution order of unseen tools but also exhibits strong reasoning capabilities for tool combination. More importantly, GeoTP also demonstrates stable performance in parameter accuracy (60.0% for single-tool tasks and 40.0% for triple-tool tasks), significantly outperforming other models in multi-tool scenarios. This further indicates that the model does not rely on rote memorization of specific tool parameters learned during training. Instead, it has acquired a generalized capability to infer parameter types, constraints, and appropriate values based on tool descriptions, enabling rapid adaptation to new tools. Compared with its performance on tasks within the training distribution (Table 5), GeoTP maintains a comparable performance level on external tool tasks, with an even more pronounced advantage in multi-tool settings. These results suggest that the instruction tuning data constructed in this study effectively help the model understand the general functional logic and parameter specifications of GIS tools, thereby equipping it with genuine generalization ability in tool usage rather than limiting it to combinations of previously seen tools. This further validates the effectiveness of the data construction method proposed in this paper and the robustness of the model.

3.4.   Case study

This section presents two real-world geospatial task cases to illustrate how GeoTP interprets users' natural language instructions and generates logically coherent and parameter-complete executable workflows. These two cases demonstrate that the model trained with the structured and parameterized instruction data (GeoITC) proposed in this paper can not only perform accurate tool planning but also achieve precise parameter inference and adaptation.

As shown in Fig. 8, this workflow fully demonstrates the entire process from road centerline discretization and endpoint elevation sampling to slope calculation. GeoTP first identifies the core objective as 'calculating road slope' and, based on geospatial knowledge, understands that road slope is the ratio of the elevation difference between the two ends of a road segment to its horizontal distance. Therefore, to calculate the slope of continuous road segments, the road centerline must first be divided into discrete segments, then the elevations of the start and end points of each segment must be obtained, and finally, the mathematical computation is performed. Specifically, GeoTP first uses the Split Lines by Maximum Length tool to divide the continuous road centerline into equal-length segments. The key parameter inference is reflected in GeoTP's ability to automatically set a reasonable segment length based on the typical requirements of road slope analysis, ensuring the precision of slope calculation. Next, to obtain the elevation of each road segment, GeoTP uses the Extract Specific Vertices tool and precisely sets the vertex indices to extract the start and end vertices of each line segment, generating a point layer that contains all endpoints. GeoTP then applies the Sample Raster Values tool to spatially associate the generated endpoints with the DEM data, sample elevation values, and assign a unified prefix to the elevation fields for differentiation. Finally, GeoTP employs the Field Calculator tool to link the sampled elevation values back to the original road segments and perform slope calculations. The capability of generating key parameters is demonstrated as GeoTP automatically creates the field calculation expression based on the slope formula: abs(('end_elev' - 'start_elev') / ${\$} $length * 100), where $length represents the geometric length of the segment, and 'end_elev' and 'start_elev' are the sampled elevation fields. This demonstrates GeoTP's capability for logical planning and parameter context inference in multi-step tasks.

Figure 8. 

Case 1: demonstration of GeoTP using four tools to calculate the road slope.

As shown in Fig. 9, this case demonstrates the complete workflow from road network data acquisition and geometric preprocessing to turning radius calculation. GeoTP first identifies the core task as 'extracting the turning radius of roads within a specified area'. Based on general principles of geospatial analysis, it plans the steps of obtaining the raw road network data, followed by geometric refinement and specialized computation. Specifically, GeoTP first employs the OSM Downloader Tool and, according to the description of the 'specified area' in the instruction, accurately maps the spatial extent of the input file rectangle_range.shp to the tool's Download Area parameter. Subsequently, GeoTP recognizes that the raw geographic coordinate data downloaded are unsuitable for distance-based geometric calculations. Therefore, it automatically inserts the Reproject Layer step to convert the data into a projected coordinate system suitable for local-scale measurements, ensuring the accuracy of subsequent computations. Next, to calculate the turning radius at each point along the road, it is necessary to generate a dense set of sampling points along the road alignment. GeoTP selects the Points Along Geometry tool and, based on common practices in road geometry analysis, sets an appropriate sampling interval to balance computational efficiency and detail resolution. Finally, GeoTP applies the Turning Radius Calculation tool to compute the turning radius for the generated path points.

Figure 9. 

Case 2: demonstration of GeoTP using four tools to calculate the road turning radius.

3.5.   Robustness analysis of the model

In practical applications, input data may contain errors or outliers. Therefore, this section focuses on examining the robustness of GeoTP when processing data containing exceptions. Specifically, for each geospatial task in GeoTP, this study randomly selects words from the input data for analysis. Characters in correct words are randomly substituted. For example, when the input is word, the system randomly selects one character in the word and substitutes it with another character, ultimately generating ward as the input result. Multiple experiments were conducted to evaluate GeoTP's performance under different levels of input exceptions, including cases in which each task contains 2, 4, 6, 8, or 10 character errors. All experimental results are presented in Fig. 10. The data show that as the number of input exceptions increases, GeoTP's accuracy gradually declines, which is largely consistent with the expectations of this study. Notably, despite the increased number of errors in the input data, GeoTP still achieves higher accuracy than the baseline models, which fully demonstrates the robustness of the proposed model.

Figure 10. 

Performance of GeoTP in processing anomalous input data on the GeoITC-Eval dataset.

3.6.   Model inference efficiency

For many geospatial applications, real-time processing capability is a critical requirement. This section evaluates the computational efficiency of GeoTP. Specifically, this paper tests the computational efficiency of GeoTP on the evaluation dataset GeoITC-Eval and records the average processing time required for tasks of different complexity levels. As shown in Table 8, the time required for GeoTP to complete a task increases with task complexity, depending on the number of tools required by each task. Although the computational efficiency of GeoTP is slightly lower than that of DeepSeek-V3.1, its performance remains within an acceptable range.

This section discusses the effect of the LoRA rank r on the performance of the GeoTP model. During training, the scale of the tunable parameter module increases with the LoRA rank r, and the dimensionality of the introduced low-rank adaptation matrices becomes larger. Theoretically, this expansion enhances the model's plasticity. To investigate the relationship between parameter scale and downstream task performance, models trained with different ranks r were systematically evaluated using the GeoITC dataset, while keeping other hyperparameters constant. The experimental results are shown in Fig. 11. Overall, under all rank r settings, the performance of GeoTP significantly surpasses that of the original unfine-tuned model and other baseline models listed in Table 5, which fully demonstrates the robustness of the proposed 'data–fine-tuning' framework. These results indicate that fine-tuning with the designed structured instruction data (GeoITC) can effectively enhance the model's core capabilities in geospatial tasks, even under varying degrees of parameter adaptation scale. Furthermore, the model's performance does not increase monotonically with a higher rank r. When r = 16, GeoTP achieved the best results in both tool-use chain and Parameter Accuracy (PA). However, when r increased to 32, the performance slightly declined. This suggests that an excessively large LoRA module size is not the key to improving performance and may even hinder the model's efficient learning of tool-use chain logic and parameter configuration rules from high-quality instruction data due to the introduction of redundant parameters. A moderate and balanced rank r provides sufficient parameter flexibility to accommodate domain-specific knowledge while avoiding overfitting, thereby achieving optimal generalization performance in downstream tasks.

Figure 11. 

Performance of GeoTP with different LoRA ranks on GeoITC-Eval.

This experiment further validates the effectiveness of the data strategy presented in this paper: by carefully constructing instruction data that emphasizes tool sequences and parameter specifications, the model can efficiently learn and master complex geospatial workflow planning and execution capabilities with relatively limited parameter adjustments.

4.2.   Impact of training data volume on model performance

This section discusses the impact of the GeoITC training dataset size on model performance. By adjusting the proportion of GeoITC data used for training, the relationship between training data volume and model performance was systematically analyzed. As shown in Fig. 12, the experimental results indicate that as the amount of training data increases, the performance of the GeoTP model exhibits a consistent upward trend in both key metrics: Chain-of-Tool Accuracy (CTOA) and Parameter Accuracy (PA).

Figure 12. 

Performance of GeoTP with different proportions of training data on GeoITC-Eval.

Specifically, when the training data volume increased from 0% to 100%, the CTOA improved from 14.5% to 81.2%, and the PA increased from 30.0% to 81.2%. This significant performance improvement validates the effectiveness of the data collection strategy proposed in this study. It demonstrates that training with structured tool-use chains data enables the model to better grasp the logical sequence and combination patterns among tools, while the parameterized instruction design helps the model learn how to configure parameters appropriately according to the task context, thereby maintaining execution accuracy in complex scenarios. It is noteworthy that even when only 25% of the training data were used, GeoTP achieved 43.6% in CTOA and 62.1% in PA, significantly outperforming the original model without fine-tuning and surpassing several baseline models (see Table 5). This fully demonstrates the robustness of the proposed framework, indicating that the designed instruction fine-tuning data can effectively convey the planning logic and parameter configuration knowledge of geospatial tasks even at a limited scale, thereby providing a feasible solution for model training in resource-constrained scenarios.

This paper proposes an effective open-source LLM-based technical framework to address three major challenges in the automatic resolution of Geospatial Tasks using LLMs: the scarcity of high-quality instruction data, the inconsistency in tool parameter configuration, and the limited performance of open-source LLMs. The research contributions are reflected in three aspects. First, a structured method is introduced for GIS tool description and parameterized instruction data generation. By incorporating a self-guidance generation strategy and leveraging domain knowledge, LLMs are guided to automatically generate diverse 'instruction–toolchain' pairs covering a wide range of scenarios and complexities. This approach particularly enhances the structured representation of parameter templates, data types, and value constraints, thereby establishing a solid foundation for the model to learn precise tool invocation logic and achieve effective Parameter Adaptation. Second, by efficiently constructing the GeoITC dataset within a simulated environment to cover scenarios involving multiple tool combinations, this study effectively addresses the scarcity of high-quality training data in the geospatial domain. GeoITC not only includes multi-level tasks such as single-tool, dual-tool, and triple-tool configurations but also emphasizes the structured representation of parameter transfer relationships among tools, providing rich supervision signals for the model to learn complex workflow logic. Furthermore, by developing a systematic evaluation benchmark, GeoITC-Eval, which incorporates multi-level task complexity and external tool generalization scenarios, the model can be comprehensively assessed in a standardized and quantifiable manner across two dimensions: chain tool operation accuracy (CTOA) and parameter accuracy (PA). This benchmark establishes a complete data foundation and evaluation framework for training and performance assessment of intelligent planning models for GIS tools. Finally, based on the LLaMA-3-8B model, the LoRA technique is applied to inject GIS tool usage capabilities, resulting in a specialized GIS tool planning model named GeoTool Planner (GeoTP). Experimental results show that GeoTP significantly outperforms baseline models across all evaluation metrics. For known tool tasks, it achieves a single-tool tool-use chain accuracy (CTOA) of 91.5% and a parameter accuracy (PA) of 90.0%. When handling tasks involving previously unseen external tools or longer tool-use chains (four to five tools), the model still maintains over 53.3% CTOA and up to 76.7% PA, demonstrating reliable generalization and strong planning ability for complex tasks. Ablation studies further confirm the critical role of structured parameter data and appropriately chosen LoRA rank in enhancing model performance.

In summary, this study not only introduces GeoTP, a high-performance and privately deployable intelligent planning tool for GIS tools, but also establishes a comprehensive methodological framework covering data generation, model training, and performance evaluation. Experimental results demonstrate that fine-tuning with carefully designed parameterized instruction data can significantly enhance open-source LLMs, enabling them to accurately master the planning logic and parameter adaptation capabilities required for complex geospatial workflows. The outstanding performance of GeoTP on known tasks, along with its strong generalization ability when handling unseen tools and extended tool-use chains, provides a solid technical foundation for developing low-cost, highly reliable, and privacy-preserving intelligent planning models for GIS tools. Although GeoTP demonstrated outstanding performance in the experiments, the evaluation results indicate that there is still room for improvement in handling extremely complex tasks, such as long chains involving multiple tools. In addition, reducing the computational overhead of model deployment and enabling dynamic learning from large-scale tool repositories remain key challenges in practical applications. Future work will focus on exploring techniques such as retrieval-augmented generation to allow the model to dynamically invoke larger toolsets without retraining, as well as optimizing the model architecture and quantization strategies to enhance inference efficiency while maintaining performance.