Use LLMs for Description or Transcription tasks
To interact with a Large Language Model (LLM) from within your nodegoat environment, you can establish a connection to the API of an LLM. You can do this via the service of a specific LLM provider like Claude or ChatGPT, or use an LLM broker like NagaAI. The advantage of an LLM broker is that you can quickly switch between providers and models.
The connection between your nodegoat environment and an LLM via an API is established by means of the 'Linked Data' module. This feature was added to nodegoat in 2015 to facilitate the connection between nodegoat and Linked Data resources. Just as you can use this feature to send the string 'Ἥφαιστος' to the SPARQL endpoint of Wikidata and receive the MANTO ID as a response, you can also use this feature to send an image to the API of an LLM and receive a description of this image in return (or receive a transcription, or generate the translation of a text, etc.).
These steps will guide you through the process of establishing the connection:
- Set up an account that gives you access to the API of an LLM.
- A Data Model in your nodegoat environment that can host data provided by you and that can host data that is generated by an LLM.
- Objects that contain data that can be send to the API of the LLM.
- A Linked Data Resource that established the connection to the API of the LLM.
- An Ingestion Process that selects the data that is send to the LLM and selects the element in the Data Model that receives the generated data.
Access to the API
Decide which LLM provider you want to use and create an account that gives you access to the API console. If you want to switch between providers and models, use an LLM broker like NagaAI. API access is almost always a paid service, so make sure you have enough credit in your account to make this work.
Follow the documentation of the provider or broker to learn how to connect to the API. NagaAI has good documentation available.
Data Model and Data
To make this process work you need an Object Type that has at least two Object Descriptions. One Object Description should contain the value send to the LLM. This value can be a string, number, date, text, media file, URL, or result of a reversal with a template. The second Object Description should be configured so that it can host the generated data. Based on the value type of the generated data, this can be a string, number, date, text, media file, or URL.
Make sure you have added at least one Object of this Type that contains a value in the Object Description that will be send to the LLM. The example used in this Guide has a Data Model that allows us to store an image and has an Object Description with the value type 'Text' that allows us to store the generated description of an image.

Linked Data Resource
Next, configure a Linked Data Resource. This resource allows you to specify a configuration that establishes a link to any external resource. This configuration can be a simple query, but can also include elements to query the external resource based on values stored in your nodegoat environment.
Go to Model and go to 'Linked Data'. Click 'Add Linked Data Resource' and give the resource a name like 'NagaAI API'. Set the protocol to 'API' and change the protocol request to 'POST'. Enter the request URL 'https://api.naga.ac/v1/chat/completions' in the 'URL' input field and leave the 'URL Options' empty. Add two 'URL Headers': 'Authorization' : 'Bearer [YOUR API KEY]', and 'Content-Type' : 'application/json'.
Follow the documentation of the API to learn how the request body needs to be formulated. In this case use the following request body:
{
"model": "qwen/qwen3.8-27b",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe this image."
},
{
"type": "image_url",
"image_url": {
"url": "8ed8630aa14ff7a834f0e1c9b29968c7.jpg"
}
}
]
}
],
"temperature": 0,
"top_p": 0.1,
"stream": false
}To be able to use a value in the request we need to specify the part of the request that can be modified when the query runs. In this case we want to provide a different image for every request, so we adjust the request by changing the filename into the following:
[query=image][variable=file:data-url]8ed8630aa14ff7a834f0e1c9b29968c7.jpg[/variable][/query]These tags do two things. The 'query' tag allows you to specify the value to be used from your nodegoat environment when you configure the Ingestion Process in the following step. The way in which the variable is configured will transform the image into a file literal. This means that the image file is send together with the request. These options are described in-depth in the Linked Data documentation section.
When working with textual data the following tags suffice: [query=text][variable]Beste Jacob, schrijf je me nog eens een briefje? Groetjes van Ferdinand.[/variable][/query].
The current request selects the model 'qwen/qwen3.8-27b'. For increased flexibility this value can also be changed into a query option so you can change the model on the level of the Ingestion Process.
The request used in the example sends a single message to the LLM. It is important to note that you can extend this in various manners by including additional messages using different 'roles' such as 'system' and 'assistant' in order to provide a better context or even employ a 'few shot prompting' approach to increase the quality of results.
Click the green 'test' button to run the query. This action will combine the values in the 'URL' field, the 'URL Headers' field and the 'Query' field and will execute the request.
After the query has run, the results are shown in the 'Response' input field. This field allows you to inspect the results. By clicking the green 'use' button, the returned data populates the mapping options.
After the 'use' button is clicked, you can select the position in the returned data that contains the values that you want to store. You can either use the value option, or use the pre-configured URI and Label options. Change the dropdown menu next to the label 'URI' and select the value {"id":""}. Change the dropdown menu next to the label 'Label' and select the value {"choices":{"[]":{"message":{"content":""}}}}.

Click 'Save Linked Data Resource' to store this configuration.
Ingestion Process
You are now ready to configure the Ingestion Process. Enable these processes by going to Management and select 'Projects'. Edit your project and enable the System Process 'Ingestion'.
Go to the Data section of your environment. Go to the tab 'Processes', click 'Ingestion', and click 'Add Ingestion'.
Give the Ingestion Process a name like 'Image Description'. Use the dropdown menu with the label 'Source' to select the Linked Data Resource 'NagaAI API'. Use the dropdown menu with the label 'Target' to select the Object Type 'Person'. Since we are updating existing Objects you need to change the 'Mode' to 'Update Existing Objects'.
Disregard the 'Query / Filter External Resource By Value' form section.
Use the form section 'Link External Resource To Object' to set 'Identify Objects By' to 'Query' and use the first dropdown menu ('Variable in the Linked Data query.') to select the variable 'image: file (data-url)'. Use the second dropdown menu ('Target Element in Data Model.') to select the 'Media' Object Description. With these settings in place, the Ingestion Process will run a query for every Object of the Type 'Person' and will assign the value of the 'Media' Object Description to the variable 'image: file (data-url)'.
Use the form section 'Map External Resource To Data Model' to connect the returned variables to elements in the selected Object Type. In this case you connect the 'Label' to the Object Description 'Image Description'.
Set 'uri' to empty values and use the red 'del' button to remove this option.

With these settings in place, the Ingestion Process will run a query for every Object of the Type 'Person' and will update the Object with the returned image description.