Initial Sheet Relay application
Publish container image / publish (push) Failing after 26s

This commit is contained in:
lzstealth committed 2026-09-17 18:54:22 +01:00
commit 854da073d4
15 files changed
+2885

No files matched your search

+6
View File
@@ -0,0 +1,6 @@
node_modules
data
.git
.gitignore
.env
npm-debug.log
+27
View File
@@ -0,0 +1,27 @@
name: Publish container image
on:
push:
branches:
- main
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-latest
env:
REGISTRY: git.lzstealth.com
IMAGE: git.lzstealth.com/lzstealth/google-sheets-to-api
steps:
- name: Check out source
uses: actions/checkout@v4
- name: Log in to Gitea container registry
env:
REGISTRY_USERNAME: ${{ secrets.PACKAGE_REGISTRY_USERNAME }}
REGISTRY_PASSWORD: ${{ secrets.PACKAGE_REGISTRY_PASSWORD }}
run: echo "$REGISTRY_PASSWORD" | docker login "$REGISTRY" --username "$REGISTRY_USERNAME" --password-stdin
- name: Build and publish image
run: |
docker build --tag "$IMAGE:$GITEA_SHA" --tag "$IMAGE:latest" .
docker push "$IMAGE:$GITEA_SHA"
docker push "$IMAGE:latest"
+3
View File
@@ -0,0 +1,3 @@
node_modules/
data/
.env
+13
View File
@@ -0,0 +1,13 @@
FROM node:22-alpine
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY --chown=node:node . .
USER node
EXPOSE 3000
CMD ["node", "server.js"]
+85
View File
@@ -0,0 +1,85 @@
# Sheet Relay
A local Node.js service that exposes selected Google Sheets tabs as bearer-token-protected JSON endpoints.
## Run with Docker
```powershell
docker login git.lzstealth.com
docker compose pull
docker compose up -d
docker compose logs -f
```
Open `http://localhost:3000`. The service is bound to localhost by default and pulls `git.lzstealth.com/lzstealth/google-sheets-to-api:latest`. Its configuration is persisted in `data/config.json` beside the Compose file, so it survives image updates and container replacements.
Pushes to `main` publish both the immutable commit tag and `latest` through Gitea Actions. The workflow uses the global `PACKAGE_REGISTRY_USERNAME` and `PACKAGE_REGISTRY_PASSWORD` secrets; grant that account permission to publish packages for the `lzstealth` organization.
Stop it with:
```powershell
docker compose down
```
## Reverse proxy deployment
The Compose service listens only at `127.0.0.1:3000`. Put your existing reverse proxy in front of it for HTTPS, public routing, access logs, and any network policy. Configure that proxy's upstream as `http://127.0.0.1:3000` and forward the original `Host`, client address, and scheme headers.
Check container health and logs with:
```powershell
docker compose ps
docker compose logs -f sheet-relay
```
## Run with Node
```powershell
npm install
npm start
```
Open `http://localhost:3000`. The first launch creates `data/config.json` and prints a one-time admin token. Keep this token private; use it to unlock the configuration console.
## Configure
1. Choose **API key** for public/readable sheets, or **Service account** for private sheets.
2. With a service account, share each spreadsheet with the `client_email` in its JSON credential as a Viewer.
3. Keep the default 45 requests per minute to leave a buffer below a 50 RPM allowance, or set the quota that applies to your Google project.
4. Load a spreadsheet ID, check the tabs to expose, then publish the endpoint. Sheet Relay generates its bearer token automatically; copy it from the endpoint controls.
## Consume
```powershell
curl http://localhost:3000/api/v1/YOUR_PUBLICATION_ID -H "Authorization: Bearer YOUR_ENDPOINT_TOKEN"
```
Only the selected tabs are returned. Responses use a configurable short cache, reducing Google API calls and respecting the shared rate limiter.
Each endpoint can return either raw sheet rows or objects keyed from the first row's headers. Object output requires non-empty, unique headers. It can also limit reads with an A1 range such as `A1:H500`, cap rows per sheet, and omit blank rows. Editing any of these options clears that endpoint's cached response immediately.
Google `429` and `5xx` responses retry up to two times. Retries honor Google's `Retry-After` header when present, otherwise use 1-second then 2-second backoff, with every retry passing through the same configured credential rate limiter.
## Operations
The **Operations** page provides service uptime, endpoint health, endpoint and tab-cache counts, and the latest 50 API request outcomes. It retains 500 entries, supports endpoint/status/result filters, and can clear the retained log. Request logs record the endpoint ID, status, cache/stale state, duration, and client address; they never store bearer tokens.
Use **Refresh** to update an endpoint cache immediately, **Disable** to return `503` without deleting its setup, or **Rotate token** to generate and copy a replacement token. Rotation invalidates the previous token immediately. Background cache refresh can be enabled per endpoint.
Use **Export redacted config** to create a transferable configuration without Google secrets, endpoint tokens, or the admin token. Importing replaces the current configuration; credentials have no secrets, endpoints receive new tokens, and endpoints remain disabled until you complete credential setup and explicitly enable them.
The public health check is available at:
```text
GET /health
```
Prometheus-compatible operational counters are available at `GET /metrics`. This endpoint requires the `x-admin-token` header.
Browser-origin requests are denied by default. Use `*` in the **Operations** page to allow all origins, or enter exact origins such as `https://app.example.com`, one per line, to allow selected browser clients. Server-to-server API clients are unaffected because they do not send an `Origin` header.
When the service is behind a reverse proxy, set **Trusted reverse proxy hops** to the number of proxies directly in front of it, typically `1`. This allows the request audit log to use the original client address from forwarded headers. Leave it at `0` unless the proxy path is trusted.
## Security notes
`data/config.json` contains Google credentials and endpoint tokens. It is intentionally ignored by Git; protect the machine and back up the file securely. Put this service behind HTTPS when it is reachable beyond localhost.
+9
View File
@@ -0,0 +1,9 @@
services:
sheet-relay:
image: git.lzstealth.com/lzstealth/google-sheets-to-api:latest
container_name: sheet-relay
ports:
- "127.0.0.1:3000:3000"
volumes:
- ./data:/app/data
restart: unless-stopped
+100
View File
@@ -0,0 +1,100 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises';
import { randomBytes } from 'node:crypto';
import path from 'node:path';
const dataDirectory = path.join(process.cwd(), 'data');
const configPath = path.join(dataDirectory, 'config.json');
const defaultConfig = () => ({
adminToken: randomBytes(24).toString('base64url'),
settings: {
corsOrigins: [],
trustedProxyHops: 0
},
credentials: [],
documents: [],
publications: []
});
function migrateConfig(config) {
if (config.credentials) {
return {
...config,
settings: {
...config.settings,
corsOrigins: config.settings?.corsOrigins || [],
trustedProxyHops: Number(config.settings?.trustedProxyHops) || 0
}
};
}
const legacyCredential = config.google?.apiKey || config.google?.serviceAccountJson ? {
id: crypto.randomUUID(),
name: 'Imported Google connection',
mode: config.google.mode,
apiKey: config.google.apiKey,
serviceAccountJson: config.google.serviceAccountJson,
requestsPerMinute: config.google.requestsPerMinute || 45
} : null;
const documents = (config.publications || []).map((publication) => ({
id: crypto.randomUUID(),
name: publication.name,
spreadsheetId: publication.spreadsheetId,
credentialId: legacyCredential?.id || '',
sheets: publication.sheets || []
}));
return {
adminToken: config.adminToken,
settings: { corsOrigins: [], trustedProxyHops: 0 },
credentials: legacyCredential ? [legacyCredential] : [],
documents,
publications: (config.publications || []).map((publication, index) => ({
id: publication.id,
name: publication.name,
documentId: documents[index].id,
sheets: publication.sheets || [],
apiToken: publication.apiToken,
cacheSeconds: config.google?.cacheSeconds || 30
}))
};
}
export async function loadConfig() {
try {
const config = migrateConfig(JSON.parse(await readFile(configPath, 'utf8')));
await saveConfig(config);
return config;
} catch (error) {
if (error.code !== 'ENOENT') throw error;
const config = defaultConfig();
await saveConfig(config);
console.log(`Created admin token: ${config.adminToken}`);
return config;
}
}
export async function saveConfig(config) {
await mkdir(dataDirectory, { recursive: true });
await writeFile(configPath, `${JSON.stringify(config, null, 2)}\n`, { mode: 0o600 });
}
export function publicConfig(config) {
return {
settings: config.settings,
credentials: config.credentials.map(({ apiKey, serviceAccountJson, ...credential }) => ({
...credential,
configured: credential.mode === 'apiKey' ? Boolean(apiKey) : Boolean(serviceAccountJson)
})),
documents: config.documents,
publications: config.publications.map(({ apiToken, ...publication }) => ({
...publication,
dataFormat: publication.dataFormat || 'rows',
range: publication.range || '',
omitBlankRows: Boolean(publication.omitBlankRows),
maxRows: Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000)),
consumerRequestsPerMinute: Math.max(1, Number(publication.consumerRequestsPerMinute) || 120),
scheduledRefresh: Boolean(publication.scheduledRefresh),
enabled: publication.enabled !== false,
tokenConfigured: Boolean(apiToken)
}))
};
}
+100
View File
@@ -0,0 +1,100 @@
import { google } from 'googleapis';
export class RateLimiter {
constructor() {
this.nextAllowedAt = 0;
}
async run(requestsPerMinute, operation) {
for (let attempt = 0; ; attempt += 1) {
const interval = Math.ceil(60_000 / Math.max(1, Number(requestsPerMinute) || 45));
const scheduledAt = Math.max(Date.now(), this.nextAllowedAt);
this.nextAllowedAt = scheduledAt + interval;
const wait = scheduledAt - Date.now();
if (wait > 0) await new Promise((resolve) => setTimeout(resolve, wait));
try {
return await operation();
} catch (error) {
const status = error.code || error.response?.status;
if (attempt >= 2 || (status !== 429 && (status < 500 || status > 599))) throw error;
const retryAfter = retryAfterMilliseconds(error);
const backoff = retryAfter ?? 1_000 * 2 ** attempt;
this.nextAllowedAt = Math.max(this.nextAllowedAt, Date.now() + backoff);
}
}
}
}
function retryAfterMilliseconds(error) {
const value = error.response?.headers?.get?.('retry-after') || error.response?.headers?.['retry-after'];
if (!value) return null;
const seconds = Number(value);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const date = Date.parse(value);
return Number.isNaN(date) ? null : Math.max(0, date - Date.now());
}
export function createSheetsClient(googleConfig) {
if (googleConfig.mode === 'serviceAccount') {
const credentials = JSON.parse(googleConfig.serviceAccountJson);
const auth = new google.auth.GoogleAuth({
credentials,
scopes: ['https://www.googleapis.com/auth/spreadsheets.readonly']
});
return google.sheets({ version: 'v4', auth });
}
if (!googleConfig.apiKey) throw new Error('A Google API key is required.');
return google.sheets({ version: 'v4', auth: googleConfig.apiKey });
}
export async function listTabs(client, spreadsheetId, limiter, requestsPerMinute) {
const response = await limiter.run(requestsPerMinute, () => client.spreadsheets.get({
spreadsheetId,
fields: 'properties.title,sheets.properties(sheetId,title)'
}));
return {
title: response.data.properties?.title ?? spreadsheetId,
sheets: (response.data.sheets ?? []).map(({ properties }) => ({
id: properties.sheetId,
title: properties.title
}))
};
}
export async function readTabs(client, publication, limiter, requestsPerMinute) {
const maxRows = Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000));
const ranges = publication.sheets.map((sheet) => {
const title = `'${sheet.title.replaceAll("'", "''")}'`;
return publication.range?.trim() ? `${title}!${publication.range.trim()}` : `${title}!1:${maxRows}`;
});
const response = await limiter.run(requestsPerMinute, () => client.spreadsheets.values.batchGet({
spreadsheetId: publication.spreadsheetId,
ranges,
majorDimension: 'ROWS'
}));
return (response.data.valueRanges ?? []).map((valueRange, index) => formatSheet(
publication.sheets[index].title,
valueRange.values ?? [],
publication
));
}
function formatSheet(sheet, values, publication) {
const maxRows = Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000));
const limitedValues = values.slice(0, maxRows);
const rows = publication.omitBlankRows
? limitedValues.filter((row) => row.some((cell) => String(cell).trim() !== ''))
: limitedValues;
if (publication.dataFormat !== 'objects') return { sheet, values: rows };
const [headers = [], ...records] = rows;
const normalizedHeaders = headers.map((header) => String(header).trim());
if (normalizedHeaders.some((header) => !header) || new Set(normalizedHeaders).size !== normalizedHeaders.length) {
throw new Error(`Object output for "${sheet}" requires non-empty, unique header names.`);
}
return {
sheet,
rows: records.map((row) => Object.fromEntries(normalizedHeaders.map((header, index) => [header, row[index] ?? ''])))
};
}
+30
View File
@@ -0,0 +1,30 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises';
import path from 'node:path';
const logPath = path.join(process.cwd(), 'data', 'request-log.json');
const maximumEntries = 500;
async function loadEntries() {
try {
return JSON.parse(await readFile(logPath, 'utf8'));
} catch (error) {
if (error.code === 'ENOENT') return [];
throw error;
}
}
export async function recordRequest(entry) {
const entries = await loadEntries();
entries.push(entry);
await mkdir(path.dirname(logPath), { recursive: true });
await writeFile(logPath, `${JSON.stringify(entries.slice(-maximumEntries), null, 2)}\n`, { mode: 0o600 });
}
export async function recentRequests(limit = 50) {
return (await loadEntries()).slice(-limit).reverse();
}
export async function clearRequests() {
await mkdir(path.dirname(logPath), { recursive: true });
await writeFile(logPath, '[]\n', { mode: 0o600 });
}
+1734
View File
File diff suppressed because it is too large. Load diff
+15
View File
@@ -0,0 +1,15 @@
{
"name": "google-sheets-to-api",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"start": "node server.js",
"dev": "node --watch server.js"
},
"dependencies": {
"express": "^5.1.0",
"googleapis": "^155.0.0",
"lucide": "^1.47.0"
}
}
+163
View File
@@ -0,0 +1,163 @@
const state = { token: sessionStorage.getItem('adminToken') || '', config: null, discoveredSheets: [], credentialId: '', documentId: '', publicationId: '' };
const secretMask = '............';
const element = (selector) => document.querySelector(selector);
const status = (message, isError = false) => { element('#status').textContent = message; element('#status').style.color = isError ? '#a83129' : '#d8562b'; };
function applyTheme(isDark) {
document.documentElement.dataset.theme = isDark ? 'dark' : '';
element('#dark-mode').checked = isDark;
}
function applyCredentialMode() {
const isApiKey = element('input[name="mode"]:checked').value === 'apiKey';
element('#api-key-field').hidden = !isApiKey;
element('#service-account-field').hidden = isApiKey;
element('#service-account-upload').hidden = isApiKey;
}
window.lucide?.createIcons();
async function adminFetch(path, options = {}) {
const response = await fetch(path, { ...options, headers: { 'content-type': 'application/json', 'x-admin-token': state.token, ...options.headers } });
if (!response.ok) throw new Error((await response.json()).error || 'Request failed.');
return response.status === 204 ? null : response.json();
}
function escapeHtml(value) { return String(value).replace(/[&<>'"]/g, (character) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', "'": '&#39;', '"': '&quot;' })[character]); }
function selectedTitles(selector) { return [...document.querySelectorAll(`${selector} input:checked`)].map((input) => input.value); }
function renderPicker(selector, sheets, selected = sheets.map((sheet) => sheet.title)) { element(selector).innerHTML = sheets.map((sheet) => `<label><input type="checkbox" value="${escapeHtml(sheet.title)}" ${selected.includes(sheet.title) ? 'checked' : ''}>${escapeHtml(sheet.title)}</label>`).join(''); }
function recordButtons(type, id) { return `<div class="actions"><button class="secondary" data-edit-${type}="${id}">Edit</button><button class="danger" data-delete-${type}="${id}">Delete</button></div>`; }
function renderCredentials() {
const credentials = state.config.credentials;
element('#credentials-list').innerHTML = credentials.length ? credentials.map((credential) => `<article class="record"><div><strong>${escapeHtml(credential.name)}</strong><br><small>${credential.mode === 'apiKey' ? 'API key' : 'Service account'} | ${credential.requestsPerMinute} RPM</small></div>${recordButtons('credential', credential.id)}</article>`).join('') : '<p class="hint">No credentials added yet.</p>';
element('#document-credential').innerHTML = credentials.length
? '<option value="" selected disabled>Select a credential profile</option>' + credentials.map((credential) => `<option value="${credential.id}">${escapeHtml(credential.name)}</option>`).join('')
: '<option value="" selected disabled>Add a credential first</option>';
}
function renderDocuments() {
const documents = state.config.documents;
element('#documents-list').innerHTML = documents.length ? documents.map((item) => {
const credential = state.config.credentials.find((entry) => entry.id === item.credentialId);
return `<article class="record"><div><strong>${escapeHtml(item.name)}</strong><br><small>${escapeHtml(credential?.name || 'Missing credential')} | ${item.sheets.map((sheet) => escapeHtml(sheet.title)).join(', ')}</small></div>${recordButtons('document', item.id)}</article>`;
}).join('') : '<p class="hint">No documents added yet.</p>';
element('#publication-document').innerHTML = documents.length
? '<option value="" selected disabled>Select a document</option>' + documents.map((item) => `<option value="${item.id}">${escapeHtml(item.name)}</option>`).join('')
: '<option value="" selected disabled>Add a document first</option>';
renderEndpointPicker();
}
function renderEndpointPicker(selected = []) {
const selectedDocument = state.config?.documents.find((item) => item.id === element('#publication-document').value);
element('#endpoint-sheet-picker').innerHTML = selectedDocument ? (() => { renderPicker('#endpoint-sheet-picker', selectedDocument.sheets, selected.length ? selected : selectedDocument.sheets.map((sheet) => sheet.title)); return element('#endpoint-sheet-picker').innerHTML; })() : '<p class="hint">Select a document to choose its tabs.</p>';
}
function renderPublications() {
const publications = state.config.publications;
element('#publications').innerHTML = publications.length ? publications.map((publication) => {
const url = `${location.origin}/api/v1/${publication.id}`;
return `<article class="endpoint"><div><strong>${escapeHtml(publication.name)}</strong><br><code>${escapeHtml(url)}</code><br><small>${publication.sheets.map((sheet) => escapeHtml(sheet.title)).join(', ')} | ${publication.consumerRequestsPerMinute} consumer RPM${publication.scheduledRefresh ? ' | background refresh' : ''}${publication.enabled ? '' : ' | disabled'}</small></div><div class="actions"><button class="secondary" data-refresh-publication="${publication.id}">Refresh</button><button class="secondary" data-toggle-publication="${publication.id}">${publication.enabled ? 'Disable' : 'Enable'}</button><button class="secondary" data-rotate-token="${publication.id}">Rotate token</button><button class="secondary" data-copy-token="${publication.id}" title="Copy bearer token">Copy token</button><button class="secondary" data-edit-publication="${publication.id}">Edit</button><button class="danger" data-delete-publication="${publication.id}">Delete</button></div></article>`;
}).join('') : '<p class="hint">No endpoints published yet.</p>';
}
function renderRequestFilters() {
const current = element('#request-endpoint').value;
element('#request-endpoint').innerHTML = '<option value="">All endpoints</option>' + state.config.publications.map((publication) => `<option value="${publication.id}">${escapeHtml(publication.name)}</option>`).join('');
element('#request-endpoint').value = current;
}
function renderRequestLog(entries) { element('#request-log').innerHTML = entries.length ? entries.map((entry) => `<article class="log-entry"><strong>${entry.status}</strong><span>${escapeHtml(entry.publicationId)}</span><span>${entry.cached ? 'cache' : 'refresh'}${entry.stale ? ', stale' : ''}</span><span>${entry.durationMs} ms</span><time>${new Date(entry.at).toLocaleString()}</time></article>`).join('') : '<p class="hint">No matching API requests.</p>'; }
async function loadConfig() { state.config = await adminFetch('/admin/config'); element('#cors-origins').value = state.config.settings.corsOrigins.join('\n'); element('#trusted-proxy-hops').value = state.config.settings.trustedProxyHops; element('#metrics-url').textContent = `${location.origin}/metrics`; renderCredentials(); renderDocuments(); renderPublications(); renderRequestFilters(); bindRecordActions(); }
async function loadRequests() { const query = new URLSearchParams(); if (element('#request-endpoint').value) query.set('publicationId', element('#request-endpoint').value); if (element('#request-status').value) query.set('status', element('#request-status').value); if (element('#request-result').value) query.set('result', element('#request-result').value); renderRequestLog(await adminFetch(`/admin/requests?${query}`)); }
async function loadStatus() {
const service = await adminFetch('/admin/status');
element('#service-status').textContent = `Healthy | ${service.uptimeSeconds}s uptime | ${service.caches.endpointEntries} endpoint cache entries | ${service.caches.tabEntries} tab cache entries`;
element('#endpoint-health').innerHTML = service.endpoints.length ? service.endpoints.map((endpoint) => `<article class="health-entry"><strong>${escapeHtml(endpoint.name)}</strong><span>${endpoint.cacheState}</span><span>${endpoint.lastSuccessAt ? `Last refresh: ${new Date(endpoint.lastSuccessAt).toLocaleString()}` : 'Not refreshed yet'}</span>${endpoint.lastError ? `<span class="health-error">${escapeHtml(endpoint.lastError)}</span>` : ''}</article>`).join('') : '<p class="hint">No endpoints published yet.</p>';
await loadRequests();
}
function showPage(page) { document.querySelectorAll('.page').forEach((section) => { section.hidden = section.id !== page; }); document.querySelectorAll('.tab').forEach((button) => button.classList.toggle('active', button.dataset.page === page)); if (page === 'operations') loadStatus().catch((error) => { element('#service-status').textContent = error.message; }); }
function resetCredential() { state.credentialId = ''; element('#credential-name').value = ''; element('#api-key').value = ''; element('#service-account').value = ''; element('#api-key').placeholder = ''; element('#service-account').placeholder = ''; element('#rpm').value = 45; element('#save-credential').textContent = 'Add credential profile'; element('#cancel-credential').hidden = true; }
function resetDocument() { state.documentId = ''; state.discoveredSheets = []; element('#document-name').value = ''; element('#document-url').value = ''; element('#sheet-picker').innerHTML = ''; element('#save-document').textContent = 'Add document'; element('#cancel-document').hidden = true; }
function resetPublication() { state.publicationId = ''; element('#publication-name').value = ''; element('#cache').value = 30; element('#max-rows').value = 10000; element('#consumer-rpm').value = 120; element('#scheduled-refresh').checked = false; element('#data-format').value = 'rows'; element('#sheet-range').value = ''; element('#omit-blank-rows').checked = false; element('#publish').textContent = 'Publish endpoint'; element('#cancel-publication').hidden = true; renderEndpointPicker(); }
function bindRecordActions() {
document.querySelectorAll('[data-edit-credential]').forEach((button) => button.addEventListener('click', () => {
const credential = state.config.credentials.find((item) => item.id === button.dataset.editCredential); state.credentialId = credential.id; element('#credential-name').value = credential.name; element('#rpm').value = credential.requestsPerMinute; element(`input[name="mode"][value="${credential.mode}"]`).checked = true; applyCredentialMode(); element('#api-key').value = credential.mode === 'apiKey' ? secretMask : ''; element('#service-account').value = credential.mode === 'serviceAccount' ? secretMask : ''; element('#save-credential').textContent = 'Save credential changes'; element('#cancel-credential').hidden = false; showPage('credentials'); status('Credential loaded. Replace the dots only to change its saved secret.');
}));
document.querySelectorAll('[data-edit-document]').forEach((button) => button.addEventListener('click', () => {
const item = state.config.documents.find((document) => document.id === button.dataset.editDocument); state.documentId = item.id; state.discoveredSheets = item.sheets; element('#document-name').value = item.name; element('#document-url').value = `https://docs.google.com/spreadsheets/d/${item.spreadsheetId}/edit`; element('#document-credential').value = item.credentialId; renderPicker('#sheet-picker', item.sheets); element('#save-document').textContent = 'Save document changes'; element('#cancel-document').hidden = false; showPage('documents'); status('Document loaded. Load sheet tabs again to refresh the available names.');
}));
document.querySelectorAll('[data-edit-publication]').forEach((button) => button.addEventListener('click', () => {
const item = state.config.publications.find((publication) => publication.id === button.dataset.editPublication); state.publicationId = item.id; element('#publication-name').value = item.name; element('#publication-document').value = item.documentId; element('#cache').value = item.cacheSeconds; element('#max-rows').value = item.maxRows; element('#consumer-rpm').value = item.consumerRequestsPerMinute; element('#scheduled-refresh').checked = Boolean(item.scheduledRefresh); element('#data-format').value = item.dataFormat || 'rows'; element('#sheet-range').value = item.range || ''; element('#omit-blank-rows').checked = Boolean(item.omitBlankRows); renderEndpointPicker(item.sheets.map((sheet) => sheet.title)); element('#publish').textContent = 'Save endpoint changes'; element('#cancel-publication').hidden = false; showPage('endpoints'); status('Endpoint loaded.');
}));
document.querySelectorAll('[data-copy-token]').forEach((button) => button.addEventListener('click', async () => {
try {
const secret = await adminFetch(`/admin/publications/${button.dataset.copyToken}/token`);
await navigator.clipboard.writeText(secret.token);
button.textContent = 'Copied';
setTimeout(() => { button.textContent = 'Copy token'; }, 1_500);
} catch (error) { status(error.message, true); }
}));
document.querySelectorAll('[data-rotate-token]').forEach((button) => button.addEventListener('click', async () => {
if (!confirm('Rotate this token? Existing clients will stop working immediately.')) return;
try {
const secret = await adminFetch(`/admin/publications/${button.dataset.rotateToken}/rotate-token`, { method: 'POST' });
await navigator.clipboard.writeText(secret.token);
button.textContent = 'New token copied';
setTimeout(() => { button.textContent = 'Rotate token'; }, 1_500);
status('Token rotated and copied.');
} catch (error) { status(error.message, true); }
}));
document.querySelectorAll('[data-refresh-publication]').forEach((button) => button.addEventListener('click', async () => {
try { await adminFetch(`/admin/publications/${button.dataset.refreshPublication}/refresh`, { method: 'POST' }); button.textContent = 'Refreshed'; setTimeout(() => { button.textContent = 'Refresh'; }, 1_500); } catch (error) { status(error.message, true); }
}));
document.querySelectorAll('[data-toggle-publication]').forEach((button) => button.addEventListener('click', async () => {
try { const publication = state.config.publications.find((item) => item.id === button.dataset.togglePublication); await adminFetch(`/admin/publications/${publication.id}/enabled`, { method: 'POST', body: JSON.stringify({ enabled: !publication.enabled }) }); await loadConfig(); } catch (error) { status(error.message, true); }
}));
['credential', 'document', 'publication'].forEach((type) => document.querySelectorAll(`[data-delete-${type}]`).forEach((button) => button.addEventListener('click', async () => { if (!confirm(`Delete this ${type}? This cannot be undone.`)) return; try { await adminFetch(`/admin/${type === 'publication' ? 'publications' : `${type}s`}/${button.dataset[`delete${type[0].toUpperCase()}${type.slice(1)}`]}`, { method: 'DELETE' }); await loadConfig(); status(`${type[0].toUpperCase()}${type.slice(1)} removed.`); } catch (error) { status(error.message, true); } })));
}
element('#connect').addEventListener('click', async () => { state.token = element('#admin-token').value.trim(); try { await loadConfig(); sessionStorage.setItem('adminToken', state.token); element('#access-panel').hidden = true; element('#console').hidden = false; status('Connected.'); } catch (error) { status(error.message, true); } });
document.querySelectorAll('.tab').forEach((button) => button.addEventListener('click', () => showPage(button.dataset.page)));
applyTheme(localStorage.getItem('theme') === 'dark');
applyCredentialMode();
document.querySelectorAll('input[name="mode"]').forEach((input) => input.addEventListener('change', applyCredentialMode));
element('#dark-mode').addEventListener('change', (event) => {
const isDark = event.target.checked;
localStorage.setItem('theme', isDark ? 'dark' : 'light');
applyTheme(isDark);
});
element('#save-settings').addEventListener('click', async () => {
try {
const corsOrigins = element('#cors-origins').value.split(/\r?\n/);
await adminFetch('/admin/settings', { method: 'PUT', body: JSON.stringify({ corsOrigins, trustedProxyHops: element('#trusted-proxy-hops').value }) });
await loadConfig();
status('CORS settings saved.');
} catch (error) { status(error.message, true); }
});
element('#service-account-file').addEventListener('change', async (event) => { const [file] = event.target.files; if (file) element('#service-account').value = await file.text(); });
element('#save-credential').addEventListener('click', async () => { try { await adminFetch('/admin/credentials', { method: 'POST', body: JSON.stringify({ id: state.credentialId || undefined, name: element('#credential-name').value, mode: element('input[name="mode"]:checked').value, apiKey: element('#api-key').value === secretMask ? '' : element('#api-key').value, serviceAccountJson: element('#service-account').value === secretMask ? '' : element('#service-account').value, requestsPerMinute: element('#rpm').value }) }); await loadConfig(); resetCredential(); applyCredentialMode(); status('Credential saved.'); } catch (error) { status(error.message, true); } });
element('#cancel-credential').addEventListener('click', resetCredential);
element('#discover').addEventListener('click', async () => { try { const result = await adminFetch('/admin/documents/discover', { method: 'POST', body: JSON.stringify({ credentialId: element('#document-credential').value, url: element('#document-url').value }) }); state.discoveredSheets = result.sheets; renderPicker('#sheet-picker', result.sheets); status(`Loaded ${result.sheets.length} tab(s) from ${result.title}.`); } catch (error) { status(error.message, true); } });
element('#save-document').addEventListener('click', async () => { try { const sheets = state.discoveredSheets.filter((sheet) => selectedTitles('#sheet-picker').includes(sheet.title)); await adminFetch('/admin/documents', { method: 'POST', body: JSON.stringify({ id: state.documentId || undefined, name: element('#document-name').value, url: element('#document-url').value, credentialId: element('#document-credential').value, sheets }) }); await loadConfig(); resetDocument(); status('Document saved.'); } catch (error) { status(error.message, true); } });
element('#cancel-document').addEventListener('click', resetDocument);
element('#publication-document').addEventListener('change', () => renderEndpointPicker());
element('#publish').addEventListener('click', async () => { try { const selectedDocument = state.config.documents.find((item) => item.id === element('#publication-document').value); if (!selectedDocument) throw new Error('Select a document first.'); const titles = selectedTitles('#endpoint-sheet-picker'); const sheets = selectedDocument.sheets.filter((sheet) => titles.includes(sheet.title)); await adminFetch('/admin/publications', { method: 'POST', body: JSON.stringify({ id: state.publicationId || undefined, name: element('#publication-name').value, documentId: selectedDocument.id, sheets, cacheSeconds: element('#cache').value, maxRows: element('#max-rows').value, consumerRequestsPerMinute: element('#consumer-rpm').value, scheduledRefresh: element('#scheduled-refresh').checked, dataFormat: element('#data-format').value, range: element('#sheet-range').value, omitBlankRows: element('#omit-blank-rows').checked }) }); await loadConfig(); resetPublication(); status('Endpoint saved.'); } catch (error) { status(error.message, true); } });
['#request-endpoint', '#request-status', '#request-result'].forEach((selector) => element(selector).addEventListener('change', () => loadRequests().catch((error) => { element('#request-log').textContent = error.message; })));
element('#clear-request-log').addEventListener('click', async () => { if (!confirm('Clear all retained API request logs?')) return; try { await adminFetch('/admin/requests', { method: 'DELETE' }); await loadRequests(); } catch (error) { status(error.message, true); } });
element('#rotate-admin-token').addEventListener('click', async () => { if (!confirm('Rotate the admin token? Other admin sessions will stop working.')) return; try { const result = await adminFetch('/admin/token/rotate', { method: 'POST' }); state.token = result.token; sessionStorage.setItem('adminToken', state.token); await navigator.clipboard.writeText(result.token); status('Admin token rotated and copied.'); } catch (error) { status(error.message, true); } });
element('#export-config').addEventListener('click', async () => { try { const exported = await adminFetch('/admin/config/export'); const url = URL.createObjectURL(new Blob([JSON.stringify(exported, null, 2)], { type: 'application/json' })); const link = document.createElement('a'); link.href = url; link.download = 'sheet-relay-config.json'; link.click(); URL.revokeObjectURL(url); } catch (error) { status(error.message, true); } });
element('#import-config').addEventListener('click', () => element('#config-file').click());
element('#config-file').addEventListener('change', async (event) => { const [file] = event.target.files; if (!file) return; try { const imported = JSON.parse(await file.text()); if (!confirm('Replace all existing configuration with this import?')) return; await adminFetch('/admin/config/import', { method: 'POST', body: JSON.stringify(imported) }); await loadConfig(); await loadStatus(); status('Configuration imported. Add credential secrets, then enable endpoints.'); } catch (error) { status(error.message, true); } finally { event.target.value = ''; } });
element('#cancel-publication').addEventListener('click', resetPublication);
if (state.token) {
loadConfig().then(() => {
element('#admin-token').value = state.token;
element('#access-panel').hidden = true;
element('#console').hidden = false;
}).catch(() => {
sessionStorage.removeItem('adminToken');
state.token = '';
});
}
+64
View File
@@ -0,0 +1,64 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Sheet Relay</title>
<link rel="stylesheet" href="/styles.css">
</head>
<body>
<main>
<header><div class="header-row"><div><p class="eyebrow">GOOGLE SHEETS TO API</p><h1>Sheet Relay</h1></div><label class="theme-toggle" title="Toggle dark mode"><input id="dark-mode" type="checkbox" aria-label="Toggle dark mode"><span class="theme-icon sun" aria-hidden="true"><i data-lucide="sun"></i></span><span class="theme-icon moon" aria-hidden="true"><i data-lucide="moon"></i></span></label></div><p>Choose exactly which spreadsheet tabs become authenticated JSON endpoints.</p></header>
<section id="access-panel" class="panel">
<h2>Admin access</h2><div class="inline"><input id="admin-token" type="password" placeholder="Admin token from server startup"><button id="connect" title="Connect to admin console">Connect</button></div><p id="status" class="status">Enter the token printed when the server first starts.</p>
</section>
<div id="console" hidden>
<nav class="tabs" aria-label="Configuration pages"><button class="tab active" data-page="credentials">Credentials</button><button class="tab" data-page="documents">Documents</button><button class="tab" data-page="endpoints">Endpoints</button><button class="tab" data-page="operations">Operations</button></nav>
<section class="page panel" id="credentials"><h2>Credential profiles</h2>
<div class="two-up"><label>Profile name<input id="credential-name" placeholder="Production service account"></label><label>Requests per minute<input id="rpm" type="number" min="1" max="10000" value="45"></label></div>
<div class="mode"><label><input type="radio" name="mode" value="apiKey" checked> API key</label><label><input type="radio" name="mode" value="serviceAccount"> Service account</label></div>
<label id="api-key-field">Google API key<input id="api-key" type="password" autocomplete="off"></label>
<label id="service-account-field">Service account JSON<textarea id="service-account" rows="5" spellcheck="false"></textarea></label>
<label id="service-account-upload" class="upload">Upload service-account JSON<input id="service-account-file" type="file" accept="application/json,.json"></label>
<button id="save-credential">Add credential profile</button><button id="cancel-credential" class="secondary" hidden>Cancel edit</button><p class="hint">45 RPM leaves room below a 50 RPM quota. API keys require publicly readable sheets. Share private sheets with the service account's client email.</p><div id="credentials-list" class="records"></div>
</section>
<section class="page panel" id="documents" hidden><h2>Google documents</h2>
<div class="two-up"><label>Document name<input id="document-name" placeholder="Warehouse inventory"></label><label>Credential profile<select id="document-credential"></select></label></div>
<label>Google Sheets URL<input id="document-url" type="url" placeholder="https://docs.google.com/spreadsheets/d/.../"></label>
<button id="discover">Load sheet tabs</button><div id="sheet-picker" class="sheet-picker"></div><button id="save-document">Add document</button><button id="cancel-document" class="secondary" hidden>Cancel edit</button><p class="hint">Paste the spreadsheet URL. Select tabs by name after they are loaded.</p><div id="documents-list" class="records"></div>
</section>
<section class="page panel" id="endpoints" hidden><h2>Authenticated endpoints</h2>
<div class="two-up"><label>Endpoint name<input id="publication-name" placeholder="Inventory feed"></label><label>Document<select id="publication-document"></select></label></div>
<label>Include sheet tabs<div id="endpoint-sheet-picker" class="sheet-picker"></div></label>
<div class="two-up"><label>Output format<select id="data-format"><option value="rows">Rows</option><option value="objects">Objects using header row</option></select></label><label>A1 range (optional)<input id="sheet-range" placeholder="A1:H500"></label></div>
<label class="check-option"><input id="omit-blank-rows" type="checkbox"> Omit blank rows</label>
<div class="two-up"><label>Cache seconds<input id="cache" type="number" min="0" max="86400" value="30"></label><label>Maximum rows per sheet<input id="max-rows" type="number" min="1" max="100000" value="10000"></label></div>
<label>Consumer requests per minute<input id="consumer-rpm" type="number" min="1" max="100000" value="120"></label>
<label class="check-option"><input id="scheduled-refresh" type="checkbox"> Background cache refresh</label>
<button id="publish">Publish endpoint</button><button id="cancel-publication" class="secondary" hidden>Cancel edit</button><p class="hint">A bearer token is generated automatically. Copy it or rotate it from the endpoint controls after publishing.</p><details class="response-format"><summary>Response format</summary><pre>{
"publication": "Inventory feed",
"document": "Warehouse inventory",
"spreadsheetId": "...",
"updatedAt": "2026-09-17T12:00:00.000Z",
"sheets": [{ "sheet": "OUTPUT", "values": [["Header"], ["Value"]] }],
"cached": false
}</pre></details><div id="publications" class="records"></div>
</section>
<section class="page panel" id="operations" hidden><h2>Operations</h2>
<label>Allowed browser origins<textarea id="cors-origins" rows="4" spellcheck="false" placeholder="*&#10;https://app.example.com"></textarea></label>
<label>Trusted reverse proxy hops<input id="trusted-proxy-hops" type="number" min="0" max="10" value="0"></label>
<button id="save-settings">Save CORS settings</button><p class="hint">Browser origins are denied by default. Use * for all origins, or enter one exact origin per line to allow selected browser clients. Server-to-server callers without an Origin header are unaffected.</p>
<p class="hint">Set proxy hops to 1 when one trusted reverse proxy is in front of this service. This makes request logs use the original client address from forwarded headers.</p>
<h3>Administration</h3><div class="actions operations-actions"><button id="rotate-admin-token" class="danger">Rotate admin token</button><button id="export-config" class="secondary">Export redacted config</button><button id="import-config" class="secondary">Import config</button><input id="config-file" type="file" accept="application/json,.json" hidden></div>
<p class="hint">Imports replace all existing configuration. Credentials are imported without their secrets, endpoints receive new tokens, and remain disabled until you explicitly enable them.</p>
<h3>Metrics</h3><div class="status-card"><code id="metrics-url"></code></div>
<h3>Service status</h3><div id="service-status" class="status-card">Loading status when this page opens.</div>
<h3>Endpoint health</h3><div id="endpoint-health" class="request-log"></div>
<h3>Recent API requests</h3><div class="request-filters"><select id="request-endpoint"><option value="">All endpoints</option></select><select id="request-status"><option value="">All statuses</option><option value="200">200</option><option value="401">401</option><option value="429">429</option><option value="502">502</option><option value="503">503</option></select><select id="request-result"><option value="">All results</option><option value="cached">Cached</option><option value="refresh">Refreshed</option><option value="stale">Stale</option></select><button id="clear-request-log" class="danger">Clear log</button></div><div id="request-log" class="request-log"></div>
</section>
</div>
</main>
<script src="/vendor/lucide/lucide.min.js"></script>
<script src="/app.js"></script>
</body>
</html>
+26
View File
@@ -0,0 +1,26 @@
@import url('https://fonts.googleapis.com/css2?family=DM+Mono:wght@400;500&family=Fraunces:opsz,wght@9..144,600;9..144,700&display=swap');
:root { --ink: #15231f; --muted: #60706a; --paper: #f6f3ea; --rule: #e8e5da; --panel: #fffdf7; --field: #fffefa; --line: #d8ddd2; --field-line: #abb8b0; --accent: #d8562b; --dark: #173c35; --button-ink: #fff; --shadow: #dfe5d8; --soft: #eaf0e7; --secondary: #b7c7bd; --secondary-hover: #91a99a; }
:root[data-theme="dark"] { --ink: #e9eee7; --muted: #b4c1b9; --paper: #17201d; --rule: #29332f; --panel: #202a26; --field: #18221e; --line: #405149; --field-line: #61766b; --accent: #ef855d; --dark: #9ec9b4; --button-ink: #15231f; --shadow: #0d1311; --soft: #293a33; --secondary: #596f64; --secondary-hover: #708a7b; }
* { box-sizing: border-box; }
body { margin: 0; color: var(--ink); background: repeating-linear-gradient(90deg, transparent 0 39px, var(--rule) 40px), var(--paper); font-family: "DM Mono", monospace; font-size: 14px; }
main { width: min(880px, calc(100% - 32px)); margin: 0 auto; padding: 64px 0; }
header { border-left: 8px solid var(--accent); padding-left: 20px; margin-bottom: 36px; }.header-row { display: flex; align-items: start; justify-content: space-between; gap: 16px; }
.eyebrow, .hint { color: var(--muted); font-size: 12px; line-height: 1.6; }
h1, h2 { font-family: Fraunces, serif; letter-spacing: 0; } h1 { font-size: 48px; margin: 3px 0 10px; } h2 { margin: 0 0 20px; font-size: 25px; }
.panel { background: var(--panel); border: 1px solid var(--line); border-radius: 6px; padding: 28px; margin: 18px 0; box-shadow: 5px 5px 0 var(--shadow); }
label { display: grid; gap: 8px; margin: 14px 0; } [hidden] { display: none !important; } input, select, textarea { width: 100%; min-height: 42px; border: 1px solid var(--field-line); border-radius: 3px; background: var(--field); padding: 10px 11px; font: inherit; color: var(--ink); } select { appearance: auto; } textarea { resize: vertical; }
button { border: 0; border-radius: 3px; background: var(--dark); color: var(--button-ink); font: 500 13px "DM Mono", monospace; padding: 11px 16px; cursor: pointer; } button:hover { background: var(--accent); color: #fff; } button.danger { background: #a83129; color: #fff; } button.secondary { background: var(--secondary); color: var(--ink); } button.secondary:hover { background: var(--secondary-hover); }.secret-input { display: flex; gap: 8px; }.secret-input input { min-width: 0; }.secret-input button { flex: 0 0 auto; }.icon-button { padding: 8px 10px; }
.panel > button + button { margin-left: 8px; }
.tabs { display: flex; gap: 1px; margin: 28px 0 -18px; }.tab { border-radius: 4px 4px 0 0; background: var(--secondary); color: var(--ink); }.tab.active { background: var(--dark); color: var(--button-ink); }#console .page { border-radius: 0 6px 6px 6px; }
.inline, .two-up, .mode { display: flex; gap: 14px; align-items: end; } .two-up > label { flex: 1; } .mode { align-items: center; } .mode label { display: flex; margin: 0; align-items: center; } .mode input { width: auto; }
.status { min-height: 20px; color: var(--accent); } .sheet-picker { display: grid; grid-template-columns: repeat(auto-fit, minmax(145px, 1fr)); gap: 8px; margin: 18px 0; }.sheet-picker label { display: flex; align-items: center; border: 1px solid var(--line); padding: 10px; margin: 0; gap: 8px; }.sheet-picker input { width: auto; }
.check-option { display: flex; align-items: center; gap: 8px; }.check-option input { width: auto; min-height: auto; }
input[type="checkbox"] { appearance: none; display: grid; place-content: center; width: 18px; min-width: 18px; height: 18px; min-height: 18px; margin: 0; padding: 0; border: 1px solid var(--field-line); border-radius: 3px; background: var(--field); cursor: pointer; } input[type="checkbox"]::after { content: ""; width: 7px; height: 4px; border: solid var(--button-ink); border-width: 0 0 2px 2px; transform: rotate(-45deg) scale(0); transform-origin: center; } input[type="checkbox"]:checked { border-color: var(--dark); background: var(--dark); } input[type="checkbox"]:checked::after { transform: rotate(-45deg) scale(1); }
.records { margin-top: 24px; }.record, .endpoint { border-top: 1px solid var(--line); padding: 16px 0; display: flex; justify-content: space-between; gap: 14px; align-items: start; }.record:first-child, .endpoint:first-child { border-top: 0; padding-top: 0; }.endpoint { flex-wrap: wrap; }.endpoint code { display: inline-block; max-width: 100%; color: var(--muted); overflow-wrap: anywhere; font-size: 11px; margin: 5px 0; }.endpoint pre { width: 100%; max-width: 620px; overflow: auto; white-space: pre-wrap; background: var(--soft); border-left: 3px solid var(--accent); padding: 12px; font-size: 11px; line-height: 1.5; }.actions { display: flex; flex-wrap: wrap; justify-content: flex-end; gap: 6px; }.actions button { padding: 8px 10px; }
.response-format { margin: 18px 0; color: var(--muted); }.response-format summary { cursor: pointer; }.response-format pre { max-width: 620px; overflow: auto; background: var(--soft); border-left: 3px solid var(--accent); padding: 12px; color: var(--ink); font-size: 11px; line-height: 1.5; }
.status-card { border-left: 3px solid var(--dark); background: var(--soft); padding: 12px; line-height: 1.5; }.request-log { margin-top: 12px; }.log-entry { display: grid; grid-template-columns: 52px minmax(0, 1fr) 90px 64px 170px; gap: 10px; align-items: center; border-top: 1px solid var(--line); padding: 10px 0; font-size: 11px; }.log-entry strong { color: var(--accent); }.log-entry span { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }.log-entry time { color: var(--muted); }.theme-toggle { display: grid; position: relative; width: 32px; height: 32px; place-items: center; margin: 4px 0; border: 1px solid var(--field-line); border-radius: 50%; background: var(--field); cursor: pointer; }.theme-toggle input { position: absolute; opacity: 0; inset: 0; margin: 0; cursor: pointer; }.theme-icon { display: grid; place-items: center; color: var(--accent); }.theme-icon svg { width: 18px; height: 18px; stroke-width: 2; }.theme-icon.moon { display: none; }:root[data-theme="dark"] .theme-icon.sun { display: none; }:root[data-theme="dark"] .theme-icon.moon { display: grid; }
.health-entry { display: grid; grid-template-columns: minmax(120px, 1fr) 68px minmax(170px, 1.5fr); gap: 10px; align-items: center; border-top: 1px solid var(--line); padding: 10px 0; font-size: 11px; }.health-entry span { color: var(--muted); }.health-entry .health-error { color: #a83129; grid-column: 1 / -1; overflow-wrap: anywhere; }
.request-filters, .operations-actions { display: flex; flex-wrap: wrap; gap: 8px; align-items: end; margin: 12px 0; }.request-filters select { width: auto; min-width: 130px; min-height: 36px; padding: 6px 8px; }.request-filters button { padding: 9px 12px; }.operations-actions { justify-content: flex-start; }
@media (max-width: 600px) { .log-entry { grid-template-columns: 45px minmax(0, 1fr); }.log-entry time { grid-column: 1 / -1; } }
@media (max-width: 600px) { main { padding: 34px 0; } h1 { font-size: 38px; }.panel { padding: 20px; }.two-up, .inline { display: grid; }.inline button { width: 100%; } }
+510
View File
@@ -0,0 +1,510 @@
import { createHash, randomBytes, randomUUID, timingSafeEqual } from 'node:crypto';
import express from 'express';
import { loadConfig, publicConfig, saveConfig } from './lib/config-store.js';
import { RateLimiter, createSheetsClient, listTabs, readTabs } from './lib/google-sheets.js';
import { clearRequests, recentRequests, recordRequest } from './lib/request-log.js';
const app = express();
const config = await loadConfig();
const limiters = new Map();
const consumerLimiters = new Map();
const cache = new Map();
const endpointHealth = new Map();
const refreshingPublications = new Map();
const tabCache = new Map();
const tabCacheMilliseconds = 5 * 60 * 1000;
const refreshIntervalMilliseconds = 5_000;
const startedAt = Date.now();
const metrics = { requests: 0, errors: 0, staleResponses: 0, refreshes: 0, refreshErrors: 0, refreshDurationMilliseconds: 0 };
app.set('trust proxy', config.settings.trustedProxyHops);
app.use(express.json({ limit: '1mb' }));
app.use('/api', (request, response, next) => {
const origin = request.get('origin');
if (!origin) return next();
if (!config.settings.corsOrigins.includes('*') && !config.settings.corsOrigins.includes(origin)) return response.status(403).json({ error: 'Origin is not permitted.' });
response.set('Access-Control-Allow-Origin', origin);
response.set('Vary', 'Origin');
response.set('Access-Control-Allow-Headers', 'Authorization, Content-Type');
response.set('Access-Control-Allow-Methods', 'GET, OPTIONS');
if (request.method === 'OPTIONS') return response.status(204).end();
next();
});
app.use((request, response, next) => {
request.startedAt = Date.now();
next();
});
app.use('/vendor/lucide', express.static('node_modules/lucide/dist/umd'));
app.use(express.static('public'));
function sameSecret(received, expected) {
if (!received || !expected) return false;
const receivedBuffer = Buffer.from(received);
const expectedBuffer = Buffer.from(expected);
return receivedBuffer.length === expectedBuffer.length && timingSafeEqual(receivedBuffer, expectedBuffer);
}
function requireAdmin(request, response, next) {
if (!sameSecret(request.get('x-admin-token'), config.adminToken)) {
return response.status(401).json({ error: 'Admin authentication required.' });
}
next();
}
function requiredText(value, label) {
if (!value?.trim()) throw new Error(`${label} is required.`);
return value.trim();
}
function spreadsheetIdFrom(value) {
const input = requiredText(value, 'Google Sheets URL');
const match = input.match(/\/spreadsheets\/d\/([a-zA-Z0-9_-]+)/);
if (match) return match[1];
if (/^[a-zA-Z0-9_-]{20,}$/.test(input)) return input;
throw new Error('Enter a valid Google Sheets URL.');
}
function credentialFor(id) {
const credential = config.credentials.find((item) => item.id === id);
if (!credential) throw new Error('The selected credential profile no longer exists.');
return credential;
}
function documentFor(id) {
const document = config.documents.find((item) => item.id === id);
if (!document) throw new Error('The selected document no longer exists.');
return document;
}
function limiterFor(credential) {
const identity = credential.mode === 'serviceAccount'
? JSON.parse(credential.serviceAccountJson).client_email
: credential.apiKey;
const key = createHash('sha256').update(`${credential.mode}:${identity}`).digest('hex');
if (!limiters.has(key)) limiters.set(key, new RateLimiter());
return limiters.get(key);
}
function consumerIsLimited(publication) {
const now = Date.now();
const windowMilliseconds = 60_000;
const limit = Math.max(1, Number(publication.consumerRequestsPerMinute) || 120);
const current = consumerLimiters.get(publication.id) || { startedAt: now, count: 0 };
if (now - current.startedAt >= windowMilliseconds) {
current.startedAt = now;
current.count = 0;
}
if (current.count >= limit) return true;
current.count += 1;
consumerLimiters.set(publication.id, current);
return false;
}
async function refreshPublication(publication) {
if (refreshingPublications.has(publication.id)) return refreshingPublications.get(publication.id);
const refresh = (async () => {
const health = endpointHealth.get(publication.id) || {};
health.lastAttemptAt = new Date().toISOString();
try {
const refreshStartedAt = Date.now();
const document = documentFor(publication.documentId);
const credential = credentialFor(document.credentialId);
const payload = {
publication: publication.name,
document: document.name,
spreadsheetId: document.spreadsheetId,
updatedAt: new Date().toISOString(),
sheets: await readTabs(createSheetsClient(credential), {
spreadsheetId: document.spreadsheetId,
sheets: publication.sheets,
dataFormat: publication.dataFormat,
range: publication.range,
omitBlankRows: publication.omitBlankRows
}, limiterFor(credential), credential.requestsPerMinute)
};
cache.set(publication.id, { payload, expiresAt: Date.now() + publication.cacheSeconds * 1000 });
endpointHealth.set(publication.id, { ...health, lastSuccessAt: new Date().toISOString(), lastError: '' });
metrics.refreshes += 1;
metrics.refreshDurationMilliseconds += Date.now() - refreshStartedAt;
return payload;
} catch (error) {
endpointHealth.set(publication.id, { ...health, lastError: error.message });
metrics.refreshErrors += 1;
throw error;
} finally {
refreshingPublications.delete(publication.id);
}
})();
refreshingPublications.set(publication.id, refresh);
return refresh;
}
function endpointStatus(publication) {
const cached = cache.get(publication.id);
const health = endpointHealth.get(publication.id) || {};
return {
id: publication.id,
name: publication.name,
enabled: publication.enabled !== false,
cacheState: !cached ? 'empty' : cached.expiresAt > Date.now() ? 'fresh' : 'stale',
cacheExpiresAt: cached ? new Date(cached.expiresAt).toISOString() : null,
...health
};
}
async function tabsFor(credential, spreadsheetId) {
const identity = credential.mode === 'serviceAccount'
? JSON.parse(credential.serviceAccountJson).client_email
: credential.apiKey;
const key = createHash('sha256').update(`${credential.mode}:${identity}:${spreadsheetId}`).digest('hex');
const existing = tabCache.get(key);
if (existing?.expiresAt > Date.now()) return existing.result;
const result = listTabs(createSheetsClient(credential), spreadsheetId, limiterFor(credential), credential.requestsPerMinute);
tabCache.set(key, { expiresAt: Date.now() + tabCacheMilliseconds, result });
try {
return await result;
} catch (error) {
tabCache.delete(key);
throw error;
}
}
function auditEndpointRequest(request, publicationId, status, details = {}) {
metrics.requests += 1;
if (status >= 400) metrics.errors += 1;
if (details.stale) metrics.staleResponses += 1;
recordRequest({
at: new Date().toISOString(),
publicationId,
status,
durationMs: Date.now() - request.startedAt,
remoteAddress: request.ip,
...details
}).catch((error) => console.error('Unable to write request log:', error));
}
app.get('/health', (request, response) => response.json({
status: 'ok',
uptimeSeconds: Math.floor((Date.now() - startedAt) / 1000),
publications: config.publications.length
}));
app.get('/metrics', requireAdmin, (request, response) => {
const averageRefreshSeconds = metrics.refreshes ? metrics.refreshDurationMilliseconds / metrics.refreshes / 1_000 : 0;
response.type('text/plain').send([
'# TYPE sheet_relay_requests_total counter',
`sheet_relay_requests_total ${metrics.requests}`,
'# TYPE sheet_relay_errors_total counter',
`sheet_relay_errors_total ${metrics.errors}`,
'# TYPE sheet_relay_stale_responses_total counter',
`sheet_relay_stale_responses_total ${metrics.staleResponses}`,
'# TYPE sheet_relay_refreshes_total counter',
`sheet_relay_refreshes_total ${metrics.refreshes}`,
'# TYPE sheet_relay_refresh_errors_total counter',
`sheet_relay_refresh_errors_total ${metrics.refreshErrors}`,
'# TYPE sheet_relay_refresh_duration_seconds gauge',
`sheet_relay_refresh_duration_seconds ${averageRefreshSeconds}`
].join('\n') + '\n');
});
app.get('/admin/config', requireAdmin, (request, response) => response.json(publicConfig(config)));
app.post('/admin/token/rotate', requireAdmin, async (request, response, next) => {
try {
config.adminToken = randomBytes(24).toString('base64url');
await saveConfig(config);
response.json({ token: config.adminToken });
} catch (error) { next(error); }
});
app.put('/admin/settings', requireAdmin, async (request, response, next) => {
try {
const corsOrigins = Array.isArray(request.body.corsOrigins)
? request.body.corsOrigins.map((origin) => origin.trim()).filter(Boolean)
: [];
const trustedProxyHops = Math.max(0, Math.min(10, Number(request.body.trustedProxyHops) || 0));
if (corsOrigins.some((origin) => origin !== '*' && !/^https?:\/\/[^/]+$/i.test(origin))) throw new Error('CORS origins must be * or an http or https origin without a path.');
config.settings.corsOrigins = [...new Set(corsOrigins)];
config.settings.trustedProxyHops = trustedProxyHops;
app.set('trust proxy', trustedProxyHops);
await saveConfig(config);
response.json(config.settings);
} catch (error) { next(error); }
});
app.get('/admin/status', requireAdmin, async (request, response, next) => {
try {
response.json({
status: 'ok',
uptimeSeconds: Math.floor((Date.now() - startedAt) / 1000),
caches: { endpointEntries: cache.size, tabEntries: tabCache.size },
endpoints: config.publications.map(endpointStatus),
recentRequests: await recentRequests()
});
} catch (error) { next(error); }
});
app.get('/admin/requests', requireAdmin, async (request, response, next) => {
try {
const requests = await recentRequests(500);
const filtered = requests.filter((entry) => {
if (request.query.publicationId && entry.publicationId !== request.query.publicationId) return false;
if (request.query.status && String(entry.status) !== String(request.query.status)) return false;
if (request.query.result === 'cached' && !entry.cached) return false;
if (request.query.result === 'refresh' && entry.cached) return false;
if (request.query.result === 'stale' && !entry.stale) return false;
return true;
});
response.json(filtered);
} catch (error) { next(error); }
});
app.delete('/admin/requests', requireAdmin, async (request, response, next) => {
try {
await clearRequests();
response.status(204).end();
} catch (error) { next(error); }
});
app.get('/admin/config/export', requireAdmin, (request, response) => {
const exported = publicConfig(config);
response.json({
version: 1,
settings: exported.settings,
credentials: exported.credentials.map(({ configured, ...credential }) => credential),
documents: exported.documents,
publications: exported.publications.map(({ tokenConfigured, ...publication }) => ({ ...publication, enabled: false }))
});
});
app.post('/admin/config/import', requireAdmin, async (request, response, next) => {
try {
const imported = request.body;
if (!imported || !Array.isArray(imported.credentials) || !Array.isArray(imported.documents) || !Array.isArray(imported.publications)) throw new Error('Choose a valid exported configuration file.');
const credentialIds = new Map();
const credentials = imported.credentials.map((credential) => {
if (!['apiKey', 'serviceAccount'].includes(credential.mode)) throw new Error('Imported credential has an invalid authentication type.');
const id = randomUUID();
credentialIds.set(credential.id, id);
return { id, name: requiredText(credential.name, 'Imported credential name'), mode: credential.mode, apiKey: '', serviceAccountJson: '', requestsPerMinute: Math.max(1, Math.min(10_000, Number(credential.requestsPerMinute) || 45)) };
});
const documentIds = new Map();
const documents = imported.documents.map((document) => {
const credentialId = credentialIds.get(document.credentialId);
if (!credentialId || !Array.isArray(document.sheets) || !document.sheets.length) throw new Error('Imported documents must reference an included credential and have at least one sheet.');
const id = randomUUID();
documentIds.set(document.id, id);
return { id, name: requiredText(document.name, 'Imported document name'), spreadsheetId: requiredText(document.spreadsheetId, 'Imported spreadsheet ID'), credentialId, sheets: document.sheets };
});
const publications = imported.publications.map((publication) => {
const documentId = documentIds.get(publication.documentId);
if (!documentId || !Array.isArray(publication.sheets) || !publication.sheets.length) throw new Error('Imported endpoints must reference an included document and have at least one sheet.');
return { id: randomUUID(), name: requiredText(publication.name, 'Imported endpoint name'), documentId, sheets: publication.sheets, apiToken: randomBytes(24).toString('base64url'), cacheSeconds: Math.max(0, Math.min(86_400, Number(publication.cacheSeconds) || 0)), dataFormat: publication.dataFormat === 'objects' ? 'objects' : 'rows', range: String(publication.range || '').trim(), omitBlankRows: Boolean(publication.omitBlankRows), maxRows: Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000)), consumerRequestsPerMinute: Math.max(1, Math.min(100_000, Number(publication.consumerRequestsPerMinute) || 120)), scheduledRefresh: Boolean(publication.scheduledRefresh), enabled: false };
});
config.settings = { corsOrigins: Array.isArray(imported.settings?.corsOrigins) ? imported.settings.corsOrigins : [], trustedProxyHops: Math.max(0, Math.min(10, Number(imported.settings?.trustedProxyHops) || 0)) };
config.credentials = credentials;
config.documents = documents;
config.publications = publications;
cache.clear();
endpointHealth.clear();
app.set('trust proxy', config.settings.trustedProxyHops);
await saveConfig(config);
response.json(publicConfig(config));
} catch (error) { next(error); }
});
app.post('/admin/credentials', requireAdmin, async (request, response, next) => {
try {
const { id, name, mode, apiKey, serviceAccountJson, requestsPerMinute } = request.body;
if (!['apiKey', 'serviceAccount'].includes(mode)) throw new Error('Choose an authentication type.');
const existing = config.credentials.find((item) => item.id === id);
const suppliedSecret = mode === 'apiKey' ? apiKey?.trim() : serviceAccountJson?.trim();
const secret = suppliedSecret || (existing?.mode === mode ? (mode === 'apiKey' ? existing.apiKey : existing.serviceAccountJson) : '');
requiredText(secret, mode === 'apiKey' ? 'Google API key' : 'Service account JSON');
if (mode === 'serviceAccount') JSON.parse(secret);
const credential = {
id: id || randomUUID(),
name: requiredText(name, 'Profile name'),
mode,
apiKey: mode === 'apiKey' ? secret : '',
serviceAccountJson: mode === 'serviceAccount' ? secret : '',
requestsPerMinute: Math.max(1, Math.min(10_000, Number(requestsPerMinute) || 45))
};
const index = config.credentials.findIndex((item) => item.id === credential.id);
if (index >= 0) config.credentials[index] = credential;
else config.credentials.push(credential);
await saveConfig(config);
response.status(201).json(publicConfig(config).credentials);
} catch (error) { next(error); }
});
app.delete('/admin/credentials/:id', requireAdmin, async (request, response, next) => {
try {
if (config.documents.some((document) => document.credentialId === request.params.id)) throw new Error('Remove or reassign documents using this profile first.');
config.credentials = config.credentials.filter((credential) => credential.id !== request.params.id);
limiters.delete(request.params.id);
await saveConfig(config);
response.status(204).end();
} catch (error) { next(error); }
});
app.post('/admin/documents/discover', requireAdmin, async (request, response, next) => {
try {
const credential = credentialFor(request.body.credentialId);
const spreadsheetId = spreadsheetIdFrom(request.body.url);
const result = await tabsFor(credential, spreadsheetId);
response.json({ ...result, spreadsheetId });
} catch (error) { next(error); }
});
app.post('/admin/documents', requireAdmin, async (request, response, next) => {
try {
const { id, name, url, credentialId, sheets } = request.body;
credentialFor(credentialId);
if (!Array.isArray(sheets) || !sheets.length) throw new Error('Select at least one sheet tab.');
const document = { id: id || randomUUID(), name: requiredText(name, 'Document name'), spreadsheetId: spreadsheetIdFrom(url), credentialId, sheets };
const index = config.documents.findIndex((item) => item.id === document.id);
if (index >= 0) config.documents[index] = document;
else config.documents.push(document);
await saveConfig(config);
response.status(201).json(publicConfig(config).documents);
} catch (error) { next(error); }
});
app.delete('/admin/documents/:id', requireAdmin, async (request, response, next) => {
try {
if (config.publications.some((publication) => publication.documentId === request.params.id)) throw new Error('Remove endpoints using this document first.');
config.documents = config.documents.filter((document) => document.id !== request.params.id);
await saveConfig(config);
response.status(204).end();
} catch (error) { next(error); }
});
app.post('/admin/publications', requireAdmin, async (request, response, next) => {
try {
const { id, name, documentId, sheets, cacheSeconds, dataFormat, range, omitBlankRows, maxRows, consumerRequestsPerMinute, scheduledRefresh } = request.body;
const document = documentFor(documentId);
const existing = config.publications.find((item) => item.id === id);
const selectedSheets = Array.isArray(sheets) && sheets.length ? sheets : document.sheets;
if (!selectedSheets.every((sheet) => document.sheets.some((available) => available.title === sheet.title))) throw new Error('Endpoint sheets must belong to the selected document.');
if (!['rows', 'objects'].includes(dataFormat || 'rows')) throw new Error('Choose a valid data format.');
const publication = { id: id || randomUUID(), name: requiredText(name, 'Endpoint name'), documentId, sheets: selectedSheets, apiToken: existing?.apiToken || randomBytes(24).toString('base64url'), cacheSeconds: Math.max(0, Math.min(86_400, Number(cacheSeconds) || 0)), dataFormat: dataFormat || 'rows', range: String(range || '').trim(), omitBlankRows: Boolean(omitBlankRows), maxRows: Math.max(1, Math.min(100_000, Number(maxRows) || 10_000)), consumerRequestsPerMinute: Math.max(1, Math.min(100_000, Number(consumerRequestsPerMinute) || 120)), scheduledRefresh: Boolean(scheduledRefresh), enabled: existing?.enabled !== false };
const index = config.publications.findIndex((item) => item.id === publication.id);
if (index >= 0) config.publications[index] = publication;
else config.publications.push(publication);
cache.delete(publication.id);
endpointHealth.delete(publication.id);
await saveConfig(config);
response.status(201).json(publicConfig(config).publications);
} catch (error) { next(error); }
});
app.delete('/admin/publications/:id', requireAdmin, async (request, response) => {
config.publications = config.publications.filter((publication) => publication.id !== request.params.id);
cache.delete(request.params.id);
consumerLimiters.delete(request.params.id);
endpointHealth.delete(request.params.id);
await saveConfig(config);
response.status(204).end();
});
app.post('/admin/publications/:id/rotate-token', requireAdmin, async (request, response, next) => {
try {
const publication = config.publications.find((item) => item.id === request.params.id);
if (!publication) throw new Error('The endpoint no longer exists.');
publication.apiToken = randomBytes(24).toString('base64url');
cache.delete(publication.id);
await saveConfig(config);
response.json({ token: publication.apiToken });
} catch (error) { next(error); }
});
app.post('/admin/publications/:id/refresh', requireAdmin, async (request, response, next) => {
try {
const publication = config.publications.find((item) => item.id === request.params.id);
if (!publication) throw new Error('The endpoint no longer exists.');
await refreshPublication(publication);
response.json(endpointStatus(publication));
} catch (error) { next(error); }
});
app.post('/admin/publications/:id/enabled', requireAdmin, async (request, response, next) => {
try {
const publication = config.publications.find((item) => item.id === request.params.id);
if (!publication) throw new Error('The endpoint no longer exists.');
publication.enabled = Boolean(request.body.enabled);
await saveConfig(config);
response.json({ enabled: publication.enabled });
} catch (error) { next(error); }
});
app.get('/api/v1/:publicationId', async (request, response, next) => {
try {
const publication = config.publications.find((item) => item.id === request.params.publicationId);
const token = request.get('authorization')?.replace(/^Bearer\s+/i, '');
if (!publication || !sameSecret(token, publication.apiToken)) {
auditEndpointRequest(request, request.params.publicationId, 401);
return response.status(401).json({ error: 'Invalid API credentials.' });
}
if (publication.enabled === false) {
auditEndpointRequest(request, publication.id, 503, { error: 'Endpoint is disabled.' });
return response.status(503).json({ error: 'This endpoint is temporarily disabled.' });
}
if (consumerIsLimited(publication)) {
auditEndpointRequest(request, publication.id, 429, { error: 'Consumer rate limit exceeded.' });
response.set('Retry-After', '60');
return response.status(429).json({ error: 'Consumer rate limit exceeded. Try again in a minute.' });
}
const cached = cache.get(publication.id);
if (cached && cached.expiresAt > Date.now()) {
auditEndpointRequest(request, publication.id, 200, { cached: true, stale: false });
return response.json({ ...cached.payload, cached: true, stale: false });
}
try {
const payload = await refreshPublication(publication);
auditEndpointRequest(request, publication.id, 200, { cached: false, stale: false });
response.json({ ...payload, cached: false, stale: false });
} catch (error) {
if (cached) {
auditEndpointRequest(request, publication.id, 200, { cached: true, stale: true, error: error.message });
return response.json({ ...cached.payload, cached: true, stale: true, staleAt: new Date(cached.expiresAt).toISOString() });
}
throw error;
}
} catch (error) {
auditEndpointRequest(request, request.params.publicationId, 502, { error: error.message });
response.status(502).json({ error: error.message || 'Unable to refresh endpoint data.' });
}
});
app.use((error, request, response, next) => {
console.error(error);
response.status(400).json({ error: error.message || 'Request failed.' });
});
const port = Number(process.env.PORT) || 3000;
app.listen(port, () => {
console.log(`Google Sheets API publisher listening on http://localhost:${port}`);
console.log('Admin token: data/config.json -> adminToken');
});
app.get('/admin/publications/:id/token', requireAdmin, (request, response, next) => {
try {
const publication = config.publications.find((item) => item.id === request.params.id);
if (!publication) throw new Error('The endpoint no longer exists.');
response.json({ token: publication.apiToken });
} catch (error) { next(error); }
});
setInterval(() => {
for (const publication of config.publications) {
if (!publication.scheduledRefresh || publication.cacheSeconds <= 0) continue;
const cached = cache.get(publication.id);
const refreshLeadMilliseconds = Math.min(10_000, publication.cacheSeconds * 200);
if (!cached || cached.expiresAt - Date.now() <= refreshLeadMilliseconds) {
refreshPublication(publication).catch((error) => console.error(`Scheduled refresh failed for ${publication.name}:`, error.message));
}
}
}, refreshIntervalMilliseconds).unref();