Fetching SUSHI usage data in an E-Resources / COUNTER 5 dataset

IMPORTANT! The COUNTER Release 5 Code of Practice is substantially different to previous releases. LibInsight supports the Platform Master, Database Master, and Title Master Reports. Please plan carefully which reports to pull into your platforms so that you do not duplicate your statistics. We advise you to retrieve reports and examine their contents to see what is included.

  • Protip: you can view the contents of any JSON file in a web browser window. Use the Filter capability to search for particular titles.
  • For example: Unique Item Requests is a metric that is valid in both the Database Master and the Title Master reports. If Platform A offers Database and Title Reports and includes the Unique Item Requests metric for Database A in both reports, those numbers will be added together, resulting in inflation of statistics.

For best results, we strongly recommend fetching only one report type per platform.

Please note: per the COUNTER Release 5 Code of Practice, "a minimum of the current year plus the prior 24 months of usage data MUST be available, unless the content provider is newly COUNTER compliant." Please check with your vendor if you are unsure how much past data they provide in COUNTER 5 format. For older data, you may need to use our COUNTER 4 datasets for E-Journals/Databases and eBooks.

Getting there

Via the command bar

To record data to a dataset, click on Record Data in the command bar. This will take you to the default dataset selected in your account's settings. If you are currently viewing the Analyze page for a dataset, however, it will take you to that same dataset's Record Data page instead.

Navigating to the Record Data page from the command bar


Via the homepage

In the Shortcuts box, select a dataset from the Record Data To dropdown.

Navigating to the Record Data page via the homepage shortcuts


Via the Datasets page

Click on a dataset's Record Data () button in the Actions column.

Navigating to the Record Data page via the Datasets page


Selecting a platform

All of the data for your e-resources are managed at the platform level.

  1. To begin adding or managing your usage and cost data, click on the platform's name or its Add Records () button in the Actions column.
  2. If you would like to clear out all of the data for a platform, click on its Empty this Platform () button in the Actions column. This will completely delete all data from the platform, so be careful!

Options for adding data or empty a platform


Manually fetching SUSHI usage data

In order for a provider to be COUNTER 5 compliant, they must provide usage data via the SUSHI protocol. As a result, there is no need to download COUNTER usage data in CSV format and upload it manually to your dataset, as there was with some COUNTER 4 providers. Instead, you can choose to fetch this data directly from your provider right from within LibInsight.

Before you can download data from one of your providers, your LibInsight admin must first add it to your list of COUNTER 5 SUSHI providers in LibInsight. This only needs to be done once for each provider.

Harvesting data

  1. Under the Usage Data tab, click on the Fetch SUSHI Data Now panel to expand it (if it isn't already).
  2. Select from one of the available SUSHI providers in your system.
    1. Note: if your SUSHI provider has an IP address restriction, see the note below the Select Provider dropdown for LibInsight's IP address. Be sure to give this address access to retrieve data on your behalf from your provider
  3. From the Report Type dropdown, select the type of COUNTER 5 report to harvest.
    • You can choose the Title Master Report, Database Master Report, or Platform Master Report. You can learn more about each report in the COUNTER Code of Practice for Release 5.
    • Keep in mind that not all service providers support all of the available COUNTER 5 reports listed in the dropdown.
    • Please plan carefully which reports to pull into your platforms so that you do not duplicate your statistics. We advise you to retrieve reports and examine their contents to see what is included.
  4. For the Usage Date Range, use the Begin Date and End Date fields to select the time period to harvest.
    • Important: because providers only allow harvesting of usage data for a maximum of a one-year date range, you will not be able to submit your request if you select a date range greater than 12 months.
    • You can not submit a request containing a future date. The maximum End Date you can select is the current month & year.
  5. Click the Get Report button.
    • If the report you're requesting is similar to a report you've already fetched, ​LibInsight will ask you to confirm whether or not you want to continue with the request. 
    • A confirmation message will appear if the fetch request was submitted successfully. The fetch itself will be processed behind the scenes on our servers. Once the fetch is complete, you'll receive an email notification.
      • Please note that some providers queue their COUNTER 5 fetch requests. We'll let you know via email if your request was queued by the provider, and again once the request goes through and is processed.
      • If there were any issues processing your request, you'll receive an email notification with additional details returned from your provider.
      • LibInsight has a maximum file size of 18 MB for fetched data files. If the data you requested exceeds that limit, you will receive an email notification that it could not be processed. Please resubmit your fetch request with a shorter date range.

Example of harvesting SUSHI data


Schedule recurring SUSHI fetches

In addition to manually fetching SUSHI data on demand, you can also schedule regular retrievals. This allows you to automatically harvest data for a platform at either monthly, quarterly, semi-annual, or yearly intervals. You can create schedules for each platform and report type; for example, you could retrieve Title data monthly and Platform data quarterly. For each schedule option, data is fetched on a different day of the month:

  • Monthly: retrieved starting on the 10th day of the following month (i.e. May's data would be fetched starting June 10)
  • Quarterly: retrieved starting the 12th of April, July, October, and January
  • Semi-annual: retrieved starting the 13th of July and January
  • Yearly: retrieved starting on January 15th of the following year (i.e. 2018 data would be fetched starting January 15, 2019)

Just as with manually fetching data, an admin must first add your SUSHI providers to LibInsight before you can create retrieval schedules.

Creating a schedule

  1. Under the Usage Data tab, click on the Schedule Future SUSHI Harvesting panel to expand it.
  2. Click on the Add New button.

The Add New button in the Schedule Future SUSHI Harvesting panel

  1. In the Add New Schedule window, use the Select Provider dropdown to select which provider to harvest from.
    • If you selected a default provider in your platform's SUSHI Credentials settings, then that provider will be selected by default.
  2. From the Report Type dropdown, select which COUNTER report you want to fetch.
  3. From the Fetch Frequency dropdown, select how often you want to harvest that report.
  4. Once you select a frequency, the Recurring Until field will appear. Click in the field to select an end date for this schedule.
  5. Optionally, you can select up to 3 users to receive email notifications once a report has been fetched.
    • Click on a name to select it.
    • To select multiple users, hold down the CTRL key and click on additional names.
  6. By default, you new schedule will be enabled. However, if you would like to delay starting this schedule, select the Disabled radio button (you can edit the schedule later to enable it).
  7. Click the Save button.

The Add New Schedule window

Managing schedules

  1. To modify a schedule, click on its Edit () button in the Actions column.
  2. To cancel a schedule, click on its Delete () button in the Actions column.

The Edit and Delete schedule buttons


Downloading fetched data

  1. Click on the Fetch List tab to view a list of all successfully fetched files. As required by the COUNTER 5 specification, usage data is provided in JSON format, which can be opened in most text or code editors.
    1. Click on the Download File () button in the Action column to download a copy of the file.
    2. If a fetched returned errors or warnings, you can view them from the link included for the fetch in the Last Fetch column.
    3. If a fetched file contains 0 records, you can click the Delete () button in the Action column to remove it. If you need to delete other files, please empty the platform's data and re-fetch the data for the needed dates. 

Example of viewing and managing fetched data

Related Articles