Schedule a daily data pull

Schedule the pull with the API

Use the same Narwhal API key for the gold request and the exchange-rate request. The script sends each request to https://api.narwhalapi.com with an Authorization: Bearer YOUR_KEY header, then appends the returned values to daily.csv. The file keeps the time of the pull alongside the gold price in USD and the USD/IDR rate.

1. Put the key in an environment file Get a free key and put export NARWHAL_API_KEY=... in ~/.narwhal.env. Set the file’s mode to 600 so only you can read it; the scheduled command sources this file before starting Python.
2. Save the daily pull script Save this as /home/me/daily_pull.py. It writes the column names when daily.csv is new and appends a row with run_at_utc, xau_usd and usd_idr after the requests succeed.
   Python, append gold and USD/IDR to a CSV

   #!/usr/bin/env python3
   """Append today's gold price and USD/IDR rate to daily.csv. Run it once a day."""
   import csv, os, pathlib
   from datetime import datetime, timezone
   import requests
   
   API = "https://api.narwhalapi.com/v1"
   H = {"Authorization": f"Bearer {os.environ['NARWHAL_API_KEY']}"}
   gold = requests.get(f"{API}/commodities/spot/XAU/USD", headers=H, timeout=10)
   fx = requests.get(f"{API}/fx/rates/USD", params={"currencies": "IDR"}, headers=H, timeout=10)
   gold.raise_for_status(); fx.raise_for_status()      # a failed run exits non-zero, so cron mail or your monitor sees it
   
   path = pathlib.Path("daily.csv")
   new = not path.exists()
   with path.open("a", newline="") as f:
       w = csv.writer(f)
       if new:
           w.writerow(["run_at_utc", "xau_usd", "usd_idr"])
       w.writerow([datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), gold.json()["price"], fx.json()["rates"][0]["rate"]])

3. Add the daily cron entry Use this crontab entry on a UTC server to schedule the pull after midnight UTC. It reads the key from the environment file and sends script output and errors to /home/me/daily_pull.log.
   Daily cron entry on a UTC server

   15 1 * * * . $HOME/.narwhal.env && /usr/bin/python3 /home/me/daily_pull.py >> /home/me/daily_pull.log 2>&1

Each request has a timeout of 10 seconds. The script reads gold from the response’s price field and USD/IDR from the rate field in rates, then writes both values in the same row. It appends to the file on later pulls, with the column header written only when the file is new.

The crontab contains the path to the environment file, while the key stays in NARWHAL_API_KEY. Narwhal uses one account, one API key and one quota across its data APIs, so the scheduled gold and exchange-rate requests share that quota. Responses carry X-RateLimit-Remaining for the remaining rate-limit allowance.

The run_at_utc column records when the script writes the row. Read it as the pull time: Free commodity and FX prices use the previous UTC day’s close. Scheduling the file append after midnight UTC keeps the daily job on the same time basis as the data.

Schedule a flow without code

A daily flow can start with n8n Schedule Trigger, Make “Schedule” or Zapier “Schedule by Zapier”. Use a Google Sheets row append to store the result. This gives the daily pull a sheet destination when you want the values in a spreadsheet instead of the script’s CSV file.

The schedule trigger starts the flow each day; the row append stores the result of that pull. Keep the timing aligned with UTC and read Free commodity and FX prices as the previous UTC day’s close. A scheduled flow follows the same daily timing as the script, with the scheduler and storage handled in the flow.

For a scheduled script, the other scheduler choices are Task Scheduler on Windows, launchd on macOS or a cloud scheduler. The daily task remains the gold request and the USD/IDR request. Use the scheduler that fits where the script will run, or use a scheduled flow when you have no server.

Which option to use when

What time zone is cron in?

Cron uses the server’s time zone. Use date to see the server time, and use UTC to match the data. The crontab entry assumes a UTC server; its after-midnight timing depends on that time zone.

FX daily rates carry a 00:00 UTC timestamp. That timestamp and the script’s run_at_utc column describe different times: the daily rate has its data timestamp, while the CSV records the time of the pull. Free commodity and FX values are the previous UTC day’s close, even when the script writes its row after midnight.

What if a daily run fails?

The script exits non-zero when a request fails, and nothing is appended. Both requests must succeed before the script opens daily.csv to append the row. A failed request therefore leaves the file without a row from that run.

Cron mail or a monitor can see the non-zero exit. Add an alert or run the script again after a failure; a commodity price alert is a related alert tutorial. The cron entry also directs output and errors to the log file named in the command.

For request-limit errors, distinguish 429 rate_limit_exceeded, which includes Retry-After, from 429 quota_exhausted, which waits for the next UTC month. Use caching and 429 handling for that part of a scheduled pull. Error responses use application/problem+json with status, code, detail and request_id.

Worked example: one month of daily pulls

Count the gold request and the USD/IDR request for each daily pull, then multiply by the days in the month. At 2 requests a day over 31 days, the script uses 62 requests, or about 1.2% of Free’s 5,000 successful requests per UTC month.

On 24 October 2026, a row with pull time 2026-10-24T12:17:15Z has gold at 3996.00 USD and USD/IDR at 18107.8. In the CSV, the pull time comes first, followed by the gold price and the exchange rate in the order set by the header.

run_at_utc,xau_usd,usd_idr
2026-10-24T12:17:15Z,3996.00,18107.8