Ingest Bibliographic Data from Zotero

This guide covers how you can ingest your Zotero data in your nodegoat environment and how you can use this connection to keep this data up to date. You can customise the configuration that is suggested in this guide based on your own needs, for example to include other fields from your Zotero library.

Follow the guide 'Add Source References' to learn how to use these references throughout your nodegoat environment.

Model

To store this bibliographic data we need to add a new Object Type. Go to Model and go to the tab 'Object Types'. Click 'Add Object Type'. Enter the name of this Type in the 'Name' field: 'Zotero Reference'.

Uncheck the 'Fixed Field' and 'In Overviews' options for the Object Name, as we will generate the Object Name based on Object Descriptions. Read the guide on creating your first Object Type to learn more about these settings.

Specify six Object Description with the name 'Authors', 'Date', 'Title', 'Publisher', 'URL' and 'Identifier'. Keep the value type of all Object Descriptions set to 'String', except 'URL': change the value type of this Object Description to 'External'. Check the 'Multiple' option for the 'Authors'  Object Description. Check the option 'Name', 'Quick Search', and 'Overview' for the first four Object Descriptions.

Click 'Save Object Type'. Go to Management and select 'Projects'. Add a new project, or edit an existing one, in order to be able to work with this new Object Type. Enable the newly created Object Type in your project.

Go to the Data section of your environment and select the Object Type you just made to check if everything has been configured correctly. Click 'Add Zotero Reference' to open the data entry form and verify that the fields that will host the new data are present.

Linked Data Resource

To learn more about how you can query your Zotero data via the Zotero API you can read the documentation. As you can see in the documentation, a request that returns references stored in your library looks like https://api.zotero.org/users/USER-ID/items/top?. You can find your USER-ID that is to be used in such a request in your Zotero settings under the heading 'Feeds/API'.

If your library is publicly accessible (i.e. you have checked the checkbox 'Publish entire library' in the 'Privacy' section of your Zotero settings), you do not need to authenticate your user when accessing your data via the API.

If your library is not publicly accessible, you need to create a new private key via the 'Feeds/API' section of your Zotero settings. Use the generated key in the next step.

Go to Model and go to 'Linked Data'. Click 'Add Linked Data Resource' and give the resource a name like 'Zotero References'. Enter the request URL 'https://api.zotero.org/users/USER-ID/items/top?' in the 'URL' input field and leave the 'URL Options' empty. If you authenticate via a private key enter 'Zotero-API-Key' in the first input field of the 'URL Headers' and your key in the second input field. If your data is publicly available, leave the 'URL Headers' empty.

We need to specify start and limit parameters to be able to correctly query the Zotero API. To do this enter the following parameters in the 'Query' input field:

limit=[[limit]]&start=[[offset]]

Click the green 'test' button to run the query. This action will combine the values in the 'URL' field and the 'Query' field.

After the query has run, the results are shown in the 'Response' input field. This field allows you to inspect the results and to verify that all specified variables have been returned. By clicking the green 'use' button, the returned data populates the mapping options.

After the 'use' button is clicked, you can select the position in the returned data that contains the URI and Label. To do this, change the dropdown menu next to the label 'URI' to: {"[]":{"data":{"key":""}}} and change the dropdown menu next to the label 'Label' to: {"[]":{"data":{"title":""}}}.

Specify four custom key/value pairs to process additional data: 'Authors', 'Publisher', 'Date', 'URL'. Change the dropdown menu for 'Authors' to: {"[]":{"data":{"creators":{"[]":{"lastName":""}}}}}, change the dropdown menu for 'Publisher' to: {"[]":{"data":{"publisher":""}}}, change the dropdown menu for 'Date' to: {"[]":{"data":{"date":""}}}, and change the dropdown menu for 'URL' to: {"[]":{"data":{"url":""}}}.

Click 'Save Linked Data Resource' to store this configuration.

Ingestion Processes

You are now ready to configure the Ingestion Process. Enable these processes by going to Management and select 'Projects'. Edit your project and enable the System Process 'Ingestion'.

Go to the Data section of your environment. Go to the tab 'Processes', click 'Ingestion', and click 'Add Ingestion'.

Give the Ingestion Process a name like 'Store Zotero References'. Use the dropdown menu with the label 'Source' to select the Linked Data Resource 'Zotero References'. Use the dropdown menu with the label 'Target' to select the Object Type 'Zotero Reference'. Change the mode of the Ingestion Process to 'Add New Objects (If Not Exists)'. This option will make sure that a next run of this process only adds new Objects that do not exist.

Disregard the form section 'Query / Filter External Resource By Value'.

In the form section 'Query External Resource By Object Value' you set the option 'Identify Objects By' to 'Filter'. Use the first dropdown menu to select the variable 'uri' and use the second dropdown menu to select the Object Description 'Identifier'. With this settings in place, the ingestion process will check if the uri in the response matches with a value stored in the 'Identifier' Object Description. If this is the case, the Object will not be added.

Disregard the form section 'Query External Resource By Object Value´.

Use the form section 'Map External Resource To Data Model' to connect the returned variables to elements in the selected Object Type. In this case you connect the variable 'uri' to the Object Description 'Identifier', you connect the variable 'label' to the Object Description 'Title', you connect the variable 'Authors' to the Object Description 'Authors', you connect the variable 'Publisher' to the Object Description 'Publisher', you connect the variable 'Date' to the Object Description 'Date', and you connect the variable 'URL' to the Object Description 'URL'.

Click 'Save Ingestion'.

You now see the newly created Ingestion Process listed in your overview of Ingestion Processes. Click the green 'run' button on the right side of this overview to run this Ingestion Process.

Click 'Run Ingestion' to run the Ingestion Process. The Ingestion Process runs and informs you about the results. If everything went correctly you now have ingested new Objects of the Type 'Zotero Reference'. Open the Type 'Zotero Reference' to see the newly ingested Objects.

To update this data you run the process again and click the red 'reset' button to clear the cached data. Newly added references will be added as Objects of the Type 'Zotero Reference'.

Follow the guide 'Modify your Object Names' to learn how to style the name of the ingested references.