MetaLocator unattended imports via SFTP

To import data via SFTP in the background on a regular basis, contact with our helpdesk is required, and a subscription to our Backgrounding addon is also required.

Configure your SFTP credentials in the import wizard by selecting SFTP as the data source.

Provide your credentials and path as shown below.

At this point, a helpdesk support technician can configure the import to repeat on the schedule you require.

Helpdesk technicians, the file must be manually uploaded via the dashboard once, with the required options.  This writes the import options and column mapping choices to a configuration file, which is referenced according to the filename used during the import .  Be careful that you use the filename, as uploaded.  Notice that MetaLocator will add a (1) to the file if an existing file exists.  The file in the URL shown below should be the file used in the “fileas” option shown below:

A crontab must then be established on that will handle the import process according to the customer-required schedule.

Login to via SSH as the schedule user using a private key. There is no password when using a private key to connect.

Enter the following command

crontab -e

Add this line to the crontab file, where the

  • user_id is the user’s user_id, the file is the remote file to download
  • file is the name of the file on the user-provided SFTP server
  • fileas is the name of the file that the remote file should be saved as locally.  This must correspond to the filename used to establish the options and mapping file in the steps above.
01 * * * * wget --keep-session-cookies --no-check-certificate " (1).csv&task=import.background_enqueue&user_id=XXXX&format=raw&Itemid=

The above crontab imports the data on a daily basis, at 1 am each day.  We prefer that files be imported in the early morning as it corresponds to the lowest global utilization.

To modify the job in the future, re-upload the user’s data and obtain the new filename.  Update the filename in the crontab URL to the new filename and the new options will be used during the next import.

To test the process before establishing a scheduled command as above, simply enter the URL in the crontab in a browser while logged into the customer account.

Files must be in CSV format.