This commit is contained in:
commit
854da073d4
15 files changed
+2885
No files matched your search
@@ -0,0 +1,6 @@
|
|||||||
|
node_modules
|
||||||
|
data
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
.env
|
||||||
|
npm-debug.log
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
name: Publish container image
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
publish:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
env:
|
||||||
|
REGISTRY: git.lzstealth.com
|
||||||
|
IMAGE: git.lzstealth.com/lzstealth/google-sheets-to-api
|
||||||
|
steps:
|
||||||
|
- name: Check out source
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
- name: Log in to Gitea container registry
|
||||||
|
env:
|
||||||
|
REGISTRY_USERNAME: ${{ secrets.PACKAGE_REGISTRY_USERNAME }}
|
||||||
|
REGISTRY_PASSWORD: ${{ secrets.PACKAGE_REGISTRY_PASSWORD }}
|
||||||
|
run: echo "$REGISTRY_PASSWORD" | docker login "$REGISTRY" --username "$REGISTRY_USERNAME" --password-stdin
|
||||||
|
- name: Build and publish image
|
||||||
|
run: |
|
||||||
|
docker build --tag "$IMAGE:$GITEA_SHA" --tag "$IMAGE:latest" .
|
||||||
|
docker push "$IMAGE:$GITEA_SHA"
|
||||||
|
docker push "$IMAGE:latest"
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
node_modules/
|
||||||
|
data/
|
||||||
|
.env
|
||||||
+13
@@ -0,0 +1,13 @@
|
|||||||
|
FROM node:22-alpine
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
COPY package.json package-lock.json ./
|
||||||
|
RUN npm ci --omit=dev
|
||||||
|
|
||||||
|
COPY --chown=node:node . .
|
||||||
|
USER node
|
||||||
|
|
||||||
|
EXPOSE 3000
|
||||||
|
|
||||||
|
CMD ["node", "server.js"]
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
# Sheet Relay
|
||||||
|
|
||||||
|
A local Node.js service that exposes selected Google Sheets tabs as bearer-token-protected JSON endpoints.
|
||||||
|
|
||||||
|
## Run with Docker
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker login git.lzstealth.com
|
||||||
|
docker compose pull
|
||||||
|
docker compose up -d
|
||||||
|
docker compose logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
Open `http://localhost:3000`. The service is bound to localhost by default and pulls `git.lzstealth.com/lzstealth/google-sheets-to-api:latest`. Its configuration is persisted in `data/config.json` beside the Compose file, so it survives image updates and container replacements.
|
||||||
|
|
||||||
|
Pushes to `main` publish both the immutable commit tag and `latest` through Gitea Actions. The workflow uses the global `PACKAGE_REGISTRY_USERNAME` and `PACKAGE_REGISTRY_PASSWORD` secrets; grant that account permission to publish packages for the `lzstealth` organization.
|
||||||
|
|
||||||
|
Stop it with:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker compose down
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reverse proxy deployment
|
||||||
|
|
||||||
|
The Compose service listens only at `127.0.0.1:3000`. Put your existing reverse proxy in front of it for HTTPS, public routing, access logs, and any network policy. Configure that proxy's upstream as `http://127.0.0.1:3000` and forward the original `Host`, client address, and scheme headers.
|
||||||
|
|
||||||
|
Check container health and logs with:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker compose ps
|
||||||
|
docker compose logs -f sheet-relay
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run with Node
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
npm install
|
||||||
|
npm start
|
||||||
|
```
|
||||||
|
|
||||||
|
Open `http://localhost:3000`. The first launch creates `data/config.json` and prints a one-time admin token. Keep this token private; use it to unlock the configuration console.
|
||||||
|
|
||||||
|
## Configure
|
||||||
|
|
||||||
|
1. Choose **API key** for public/readable sheets, or **Service account** for private sheets.
|
||||||
|
2. With a service account, share each spreadsheet with the `client_email` in its JSON credential as a Viewer.
|
||||||
|
3. Keep the default 45 requests per minute to leave a buffer below a 50 RPM allowance, or set the quota that applies to your Google project.
|
||||||
|
4. Load a spreadsheet ID, check the tabs to expose, then publish the endpoint. Sheet Relay generates its bearer token automatically; copy it from the endpoint controls.
|
||||||
|
|
||||||
|
## Consume
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
curl http://localhost:3000/api/v1/YOUR_PUBLICATION_ID -H "Authorization: Bearer YOUR_ENDPOINT_TOKEN"
|
||||||
|
```
|
||||||
|
|
||||||
|
Only the selected tabs are returned. Responses use a configurable short cache, reducing Google API calls and respecting the shared rate limiter.
|
||||||
|
|
||||||
|
Each endpoint can return either raw sheet rows or objects keyed from the first row's headers. Object output requires non-empty, unique headers. It can also limit reads with an A1 range such as `A1:H500`, cap rows per sheet, and omit blank rows. Editing any of these options clears that endpoint's cached response immediately.
|
||||||
|
|
||||||
|
Google `429` and `5xx` responses retry up to two times. Retries honor Google's `Retry-After` header when present, otherwise use 1-second then 2-second backoff, with every retry passing through the same configured credential rate limiter.
|
||||||
|
|
||||||
|
## Operations
|
||||||
|
|
||||||
|
The **Operations** page provides service uptime, endpoint health, endpoint and tab-cache counts, and the latest 50 API request outcomes. It retains 500 entries, supports endpoint/status/result filters, and can clear the retained log. Request logs record the endpoint ID, status, cache/stale state, duration, and client address; they never store bearer tokens.
|
||||||
|
|
||||||
|
Use **Refresh** to update an endpoint cache immediately, **Disable** to return `503` without deleting its setup, or **Rotate token** to generate and copy a replacement token. Rotation invalidates the previous token immediately. Background cache refresh can be enabled per endpoint.
|
||||||
|
|
||||||
|
Use **Export redacted config** to create a transferable configuration without Google secrets, endpoint tokens, or the admin token. Importing replaces the current configuration; credentials have no secrets, endpoints receive new tokens, and endpoints remain disabled until you complete credential setup and explicitly enable them.
|
||||||
|
|
||||||
|
The public health check is available at:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /health
|
||||||
|
```
|
||||||
|
|
||||||
|
Prometheus-compatible operational counters are available at `GET /metrics`. This endpoint requires the `x-admin-token` header.
|
||||||
|
|
||||||
|
Browser-origin requests are denied by default. Use `*` in the **Operations** page to allow all origins, or enter exact origins such as `https://app.example.com`, one per line, to allow selected browser clients. Server-to-server API clients are unaffected because they do not send an `Origin` header.
|
||||||
|
|
||||||
|
When the service is behind a reverse proxy, set **Trusted reverse proxy hops** to the number of proxies directly in front of it, typically `1`. This allows the request audit log to use the original client address from forwarded headers. Leave it at `0` unless the proxy path is trusted.
|
||||||
|
|
||||||
|
## Security notes
|
||||||
|
|
||||||
|
`data/config.json` contains Google credentials and endpoint tokens. It is intentionally ignored by Git; protect the machine and back up the file securely. Put this service behind HTTPS when it is reachable beyond localhost.
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
services:
|
||||||
|
sheet-relay:
|
||||||
|
image: git.lzstealth.com/lzstealth/google-sheets-to-api:latest
|
||||||
|
container_name: sheet-relay
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:3000:3000"
|
||||||
|
volumes:
|
||||||
|
- ./data:/app/data
|
||||||
|
restart: unless-stopped
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
||||||
|
import { randomBytes } from 'node:crypto';
|
||||||
|
import path from 'node:path';
|
||||||
|
|
||||||
|
const dataDirectory = path.join(process.cwd(), 'data');
|
||||||
|
const configPath = path.join(dataDirectory, 'config.json');
|
||||||
|
|
||||||
|
const defaultConfig = () => ({
|
||||||
|
adminToken: randomBytes(24).toString('base64url'),
|
||||||
|
settings: {
|
||||||
|
corsOrigins: [],
|
||||||
|
trustedProxyHops: 0
|
||||||
|
},
|
||||||
|
credentials: [],
|
||||||
|
documents: [],
|
||||||
|
publications: []
|
||||||
|
});
|
||||||
|
|
||||||
|
function migrateConfig(config) {
|
||||||
|
if (config.credentials) {
|
||||||
|
return {
|
||||||
|
...config,
|
||||||
|
settings: {
|
||||||
|
...config.settings,
|
||||||
|
corsOrigins: config.settings?.corsOrigins || [],
|
||||||
|
trustedProxyHops: Number(config.settings?.trustedProxyHops) || 0
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const legacyCredential = config.google?.apiKey || config.google?.serviceAccountJson ? {
|
||||||
|
id: crypto.randomUUID(),
|
||||||
|
name: 'Imported Google connection',
|
||||||
|
mode: config.google.mode,
|
||||||
|
apiKey: config.google.apiKey,
|
||||||
|
serviceAccountJson: config.google.serviceAccountJson,
|
||||||
|
requestsPerMinute: config.google.requestsPerMinute || 45
|
||||||
|
} : null;
|
||||||
|
const documents = (config.publications || []).map((publication) => ({
|
||||||
|
id: crypto.randomUUID(),
|
||||||
|
name: publication.name,
|
||||||
|
spreadsheetId: publication.spreadsheetId,
|
||||||
|
credentialId: legacyCredential?.id || '',
|
||||||
|
sheets: publication.sheets || []
|
||||||
|
}));
|
||||||
|
return {
|
||||||
|
adminToken: config.adminToken,
|
||||||
|
settings: { corsOrigins: [], trustedProxyHops: 0 },
|
||||||
|
credentials: legacyCredential ? [legacyCredential] : [],
|
||||||
|
documents,
|
||||||
|
publications: (config.publications || []).map((publication, index) => ({
|
||||||
|
id: publication.id,
|
||||||
|
name: publication.name,
|
||||||
|
documentId: documents[index].id,
|
||||||
|
sheets: publication.sheets || [],
|
||||||
|
apiToken: publication.apiToken,
|
||||||
|
cacheSeconds: config.google?.cacheSeconds || 30
|
||||||
|
}))
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function loadConfig() {
|
||||||
|
try {
|
||||||
|
const config = migrateConfig(JSON.parse(await readFile(configPath, 'utf8')));
|
||||||
|
await saveConfig(config);
|
||||||
|
return config;
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code !== 'ENOENT') throw error;
|
||||||
|
const config = defaultConfig();
|
||||||
|
await saveConfig(config);
|
||||||
|
console.log(`Created admin token: ${config.adminToken}`);
|
||||||
|
return config;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function saveConfig(config) {
|
||||||
|
await mkdir(dataDirectory, { recursive: true });
|
||||||
|
await writeFile(configPath, `${JSON.stringify(config, null, 2)}\n`, { mode: 0o600 });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function publicConfig(config) {
|
||||||
|
return {
|
||||||
|
settings: config.settings,
|
||||||
|
credentials: config.credentials.map(({ apiKey, serviceAccountJson, ...credential }) => ({
|
||||||
|
...credential,
|
||||||
|
configured: credential.mode === 'apiKey' ? Boolean(apiKey) : Boolean(serviceAccountJson)
|
||||||
|
})),
|
||||||
|
documents: config.documents,
|
||||||
|
publications: config.publications.map(({ apiToken, ...publication }) => ({
|
||||||
|
...publication,
|
||||||
|
dataFormat: publication.dataFormat || 'rows',
|
||||||
|
range: publication.range || '',
|
||||||
|
omitBlankRows: Boolean(publication.omitBlankRows),
|
||||||
|
maxRows: Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000)),
|
||||||
|
consumerRequestsPerMinute: Math.max(1, Number(publication.consumerRequestsPerMinute) || 120),
|
||||||
|
scheduledRefresh: Boolean(publication.scheduledRefresh),
|
||||||
|
enabled: publication.enabled !== false,
|
||||||
|
tokenConfigured: Boolean(apiToken)
|
||||||
|
}))
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
import { google } from 'googleapis';
|
||||||
|
|
||||||
|
export class RateLimiter {
|
||||||
|
constructor() {
|
||||||
|
this.nextAllowedAt = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
async run(requestsPerMinute, operation) {
|
||||||
|
for (let attempt = 0; ; attempt += 1) {
|
||||||
|
const interval = Math.ceil(60_000 / Math.max(1, Number(requestsPerMinute) || 45));
|
||||||
|
const scheduledAt = Math.max(Date.now(), this.nextAllowedAt);
|
||||||
|
this.nextAllowedAt = scheduledAt + interval;
|
||||||
|
const wait = scheduledAt - Date.now();
|
||||||
|
if (wait > 0) await new Promise((resolve) => setTimeout(resolve, wait));
|
||||||
|
try {
|
||||||
|
return await operation();
|
||||||
|
} catch (error) {
|
||||||
|
const status = error.code || error.response?.status;
|
||||||
|
if (attempt >= 2 || (status !== 429 && (status < 500 || status > 599))) throw error;
|
||||||
|
const retryAfter = retryAfterMilliseconds(error);
|
||||||
|
const backoff = retryAfter ?? 1_000 * 2 ** attempt;
|
||||||
|
this.nextAllowedAt = Math.max(this.nextAllowedAt, Date.now() + backoff);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function retryAfterMilliseconds(error) {
|
||||||
|
const value = error.response?.headers?.get?.('retry-after') || error.response?.headers?.['retry-after'];
|
||||||
|
if (!value) return null;
|
||||||
|
const seconds = Number(value);
|
||||||
|
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
|
||||||
|
const date = Date.parse(value);
|
||||||
|
return Number.isNaN(date) ? null : Math.max(0, date - Date.now());
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createSheetsClient(googleConfig) {
|
||||||
|
if (googleConfig.mode === 'serviceAccount') {
|
||||||
|
const credentials = JSON.parse(googleConfig.serviceAccountJson);
|
||||||
|
const auth = new google.auth.GoogleAuth({
|
||||||
|
credentials,
|
||||||
|
scopes: ['https://www.googleapis.com/auth/spreadsheets.readonly']
|
||||||
|
});
|
||||||
|
return google.sheets({ version: 'v4', auth });
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!googleConfig.apiKey) throw new Error('A Google API key is required.');
|
||||||
|
return google.sheets({ version: 'v4', auth: googleConfig.apiKey });
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function listTabs(client, spreadsheetId, limiter, requestsPerMinute) {
|
||||||
|
const response = await limiter.run(requestsPerMinute, () => client.spreadsheets.get({
|
||||||
|
spreadsheetId,
|
||||||
|
fields: 'properties.title,sheets.properties(sheetId,title)'
|
||||||
|
}));
|
||||||
|
return {
|
||||||
|
title: response.data.properties?.title ?? spreadsheetId,
|
||||||
|
sheets: (response.data.sheets ?? []).map(({ properties }) => ({
|
||||||
|
id: properties.sheetId,
|
||||||
|
title: properties.title
|
||||||
|
}))
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function readTabs(client, publication, limiter, requestsPerMinute) {
|
||||||
|
const maxRows = Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000));
|
||||||
|
const ranges = publication.sheets.map((sheet) => {
|
||||||
|
const title = `'${sheet.title.replaceAll("'", "''")}'`;
|
||||||
|
return publication.range?.trim() ? `${title}!${publication.range.trim()}` : `${title}!1:${maxRows}`;
|
||||||
|
});
|
||||||
|
const response = await limiter.run(requestsPerMinute, () => client.spreadsheets.values.batchGet({
|
||||||
|
spreadsheetId: publication.spreadsheetId,
|
||||||
|
ranges,
|
||||||
|
majorDimension: 'ROWS'
|
||||||
|
}));
|
||||||
|
|
||||||
|
return (response.data.valueRanges ?? []).map((valueRange, index) => formatSheet(
|
||||||
|
publication.sheets[index].title,
|
||||||
|
valueRange.values ?? [],
|
||||||
|
publication
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatSheet(sheet, values, publication) {
|
||||||
|
const maxRows = Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000));
|
||||||
|
const limitedValues = values.slice(0, maxRows);
|
||||||
|
const rows = publication.omitBlankRows
|
||||||
|
? limitedValues.filter((row) => row.some((cell) => String(cell).trim() !== ''))
|
||||||
|
: limitedValues;
|
||||||
|
if (publication.dataFormat !== 'objects') return { sheet, values: rows };
|
||||||
|
const [headers = [], ...records] = rows;
|
||||||
|
const normalizedHeaders = headers.map((header) => String(header).trim());
|
||||||
|
if (normalizedHeaders.some((header) => !header) || new Set(normalizedHeaders).size !== normalizedHeaders.length) {
|
||||||
|
throw new Error(`Object output for "${sheet}" requires non-empty, unique header names.`);
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
sheet,
|
||||||
|
rows: records.map((row) => Object.fromEntries(normalizedHeaders.map((header, index) => [header, row[index] ?? ''])))
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
||||||
|
import path from 'node:path';
|
||||||
|
|
||||||
|
const logPath = path.join(process.cwd(), 'data', 'request-log.json');
|
||||||
|
const maximumEntries = 500;
|
||||||
|
|
||||||
|
async function loadEntries() {
|
||||||
|
try {
|
||||||
|
return JSON.parse(await readFile(logPath, 'utf8'));
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code === 'ENOENT') return [];
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function recordRequest(entry) {
|
||||||
|
const entries = await loadEntries();
|
||||||
|
entries.push(entry);
|
||||||
|
await mkdir(path.dirname(logPath), { recursive: true });
|
||||||
|
await writeFile(logPath, `${JSON.stringify(entries.slice(-maximumEntries), null, 2)}\n`, { mode: 0o600 });
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function recentRequests(limit = 50) {
|
||||||
|
return (await loadEntries()).slice(-limit).reverse();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function clearRequests() {
|
||||||
|
await mkdir(path.dirname(logPath), { recursive: true });
|
||||||
|
await writeFile(logPath, '[]\n', { mode: 0o600 });
|
||||||
|
}
|
||||||
Generated
+1734
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"name": "google-sheets-to-api",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"start": "node server.js",
|
||||||
|
"dev": "node --watch server.js"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"express": "^5.1.0",
|
||||||
|
"googleapis": "^155.0.0",
|
||||||
|
"lucide": "^1.47.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
+163
@@ -0,0 +1,163 @@
|
|||||||
|
const state = { token: sessionStorage.getItem('adminToken') || '', config: null, discoveredSheets: [], credentialId: '', documentId: '', publicationId: '' };
|
||||||
|
const secretMask = '............';
|
||||||
|
const element = (selector) => document.querySelector(selector);
|
||||||
|
const status = (message, isError = false) => { element('#status').textContent = message; element('#status').style.color = isError ? '#a83129' : '#d8562b'; };
|
||||||
|
|
||||||
|
function applyTheme(isDark) {
|
||||||
|
document.documentElement.dataset.theme = isDark ? 'dark' : '';
|
||||||
|
element('#dark-mode').checked = isDark;
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyCredentialMode() {
|
||||||
|
const isApiKey = element('input[name="mode"]:checked').value === 'apiKey';
|
||||||
|
element('#api-key-field').hidden = !isApiKey;
|
||||||
|
element('#service-account-field').hidden = isApiKey;
|
||||||
|
element('#service-account-upload').hidden = isApiKey;
|
||||||
|
}
|
||||||
|
|
||||||
|
window.lucide?.createIcons();
|
||||||
|
|
||||||
|
async function adminFetch(path, options = {}) {
|
||||||
|
const response = await fetch(path, { ...options, headers: { 'content-type': 'application/json', 'x-admin-token': state.token, ...options.headers } });
|
||||||
|
if (!response.ok) throw new Error((await response.json()).error || 'Request failed.');
|
||||||
|
return response.status === 204 ? null : response.json();
|
||||||
|
}
|
||||||
|
|
||||||
|
function escapeHtml(value) { return String(value).replace(/[&<>'"]/g, (character) => ({ '&': '&', '<': '<', '>': '>', "'": ''', '"': '"' })[character]); }
|
||||||
|
function selectedTitles(selector) { return [...document.querySelectorAll(`${selector} input:checked`)].map((input) => input.value); }
|
||||||
|
function renderPicker(selector, sheets, selected = sheets.map((sheet) => sheet.title)) { element(selector).innerHTML = sheets.map((sheet) => `<label><input type="checkbox" value="${escapeHtml(sheet.title)}" ${selected.includes(sheet.title) ? 'checked' : ''}>${escapeHtml(sheet.title)}</label>`).join(''); }
|
||||||
|
function recordButtons(type, id) { return `<div class="actions"><button class="secondary" data-edit-${type}="${id}">Edit</button><button class="danger" data-delete-${type}="${id}">Delete</button></div>`; }
|
||||||
|
|
||||||
|
function renderCredentials() {
|
||||||
|
const credentials = state.config.credentials;
|
||||||
|
element('#credentials-list').innerHTML = credentials.length ? credentials.map((credential) => `<article class="record"><div><strong>${escapeHtml(credential.name)}</strong><br><small>${credential.mode === 'apiKey' ? 'API key' : 'Service account'} | ${credential.requestsPerMinute} RPM</small></div>${recordButtons('credential', credential.id)}</article>`).join('') : '<p class="hint">No credentials added yet.</p>';
|
||||||
|
element('#document-credential').innerHTML = credentials.length
|
||||||
|
? '<option value="" selected disabled>Select a credential profile</option>' + credentials.map((credential) => `<option value="${credential.id}">${escapeHtml(credential.name)}</option>`).join('')
|
||||||
|
: '<option value="" selected disabled>Add a credential first</option>';
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderDocuments() {
|
||||||
|
const documents = state.config.documents;
|
||||||
|
element('#documents-list').innerHTML = documents.length ? documents.map((item) => {
|
||||||
|
const credential = state.config.credentials.find((entry) => entry.id === item.credentialId);
|
||||||
|
return `<article class="record"><div><strong>${escapeHtml(item.name)}</strong><br><small>${escapeHtml(credential?.name || 'Missing credential')} | ${item.sheets.map((sheet) => escapeHtml(sheet.title)).join(', ')}</small></div>${recordButtons('document', item.id)}</article>`;
|
||||||
|
}).join('') : '<p class="hint">No documents added yet.</p>';
|
||||||
|
element('#publication-document').innerHTML = documents.length
|
||||||
|
? '<option value="" selected disabled>Select a document</option>' + documents.map((item) => `<option value="${item.id}">${escapeHtml(item.name)}</option>`).join('')
|
||||||
|
: '<option value="" selected disabled>Add a document first</option>';
|
||||||
|
renderEndpointPicker();
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderEndpointPicker(selected = []) {
|
||||||
|
const selectedDocument = state.config?.documents.find((item) => item.id === element('#publication-document').value);
|
||||||
|
element('#endpoint-sheet-picker').innerHTML = selectedDocument ? (() => { renderPicker('#endpoint-sheet-picker', selectedDocument.sheets, selected.length ? selected : selectedDocument.sheets.map((sheet) => sheet.title)); return element('#endpoint-sheet-picker').innerHTML; })() : '<p class="hint">Select a document to choose its tabs.</p>';
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderPublications() {
|
||||||
|
const publications = state.config.publications;
|
||||||
|
element('#publications').innerHTML = publications.length ? publications.map((publication) => {
|
||||||
|
const url = `${location.origin}/api/v1/${publication.id}`;
|
||||||
|
return `<article class="endpoint"><div><strong>${escapeHtml(publication.name)}</strong><br><code>${escapeHtml(url)}</code><br><small>${publication.sheets.map((sheet) => escapeHtml(sheet.title)).join(', ')} | ${publication.consumerRequestsPerMinute} consumer RPM${publication.scheduledRefresh ? ' | background refresh' : ''}${publication.enabled ? '' : ' | disabled'}</small></div><div class="actions"><button class="secondary" data-refresh-publication="${publication.id}">Refresh</button><button class="secondary" data-toggle-publication="${publication.id}">${publication.enabled ? 'Disable' : 'Enable'}</button><button class="secondary" data-rotate-token="${publication.id}">Rotate token</button><button class="secondary" data-copy-token="${publication.id}" title="Copy bearer token">Copy token</button><button class="secondary" data-edit-publication="${publication.id}">Edit</button><button class="danger" data-delete-publication="${publication.id}">Delete</button></div></article>`;
|
||||||
|
}).join('') : '<p class="hint">No endpoints published yet.</p>';
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderRequestFilters() {
|
||||||
|
const current = element('#request-endpoint').value;
|
||||||
|
element('#request-endpoint').innerHTML = '<option value="">All endpoints</option>' + state.config.publications.map((publication) => `<option value="${publication.id}">${escapeHtml(publication.name)}</option>`).join('');
|
||||||
|
element('#request-endpoint').value = current;
|
||||||
|
}
|
||||||
|
function renderRequestLog(entries) { element('#request-log').innerHTML = entries.length ? entries.map((entry) => `<article class="log-entry"><strong>${entry.status}</strong><span>${escapeHtml(entry.publicationId)}</span><span>${entry.cached ? 'cache' : 'refresh'}${entry.stale ? ', stale' : ''}</span><span>${entry.durationMs} ms</span><time>${new Date(entry.at).toLocaleString()}</time></article>`).join('') : '<p class="hint">No matching API requests.</p>'; }
|
||||||
|
async function loadConfig() { state.config = await adminFetch('/admin/config'); element('#cors-origins').value = state.config.settings.corsOrigins.join('\n'); element('#trusted-proxy-hops').value = state.config.settings.trustedProxyHops; element('#metrics-url').textContent = `${location.origin}/metrics`; renderCredentials(); renderDocuments(); renderPublications(); renderRequestFilters(); bindRecordActions(); }
|
||||||
|
async function loadRequests() { const query = new URLSearchParams(); if (element('#request-endpoint').value) query.set('publicationId', element('#request-endpoint').value); if (element('#request-status').value) query.set('status', element('#request-status').value); if (element('#request-result').value) query.set('result', element('#request-result').value); renderRequestLog(await adminFetch(`/admin/requests?${query}`)); }
|
||||||
|
async function loadStatus() {
|
||||||
|
const service = await adminFetch('/admin/status');
|
||||||
|
element('#service-status').textContent = `Healthy | ${service.uptimeSeconds}s uptime | ${service.caches.endpointEntries} endpoint cache entries | ${service.caches.tabEntries} tab cache entries`;
|
||||||
|
element('#endpoint-health').innerHTML = service.endpoints.length ? service.endpoints.map((endpoint) => `<article class="health-entry"><strong>${escapeHtml(endpoint.name)}</strong><span>${endpoint.cacheState}</span><span>${endpoint.lastSuccessAt ? `Last refresh: ${new Date(endpoint.lastSuccessAt).toLocaleString()}` : 'Not refreshed yet'}</span>${endpoint.lastError ? `<span class="health-error">${escapeHtml(endpoint.lastError)}</span>` : ''}</article>`).join('') : '<p class="hint">No endpoints published yet.</p>';
|
||||||
|
await loadRequests();
|
||||||
|
}
|
||||||
|
function showPage(page) { document.querySelectorAll('.page').forEach((section) => { section.hidden = section.id !== page; }); document.querySelectorAll('.tab').forEach((button) => button.classList.toggle('active', button.dataset.page === page)); if (page === 'operations') loadStatus().catch((error) => { element('#service-status').textContent = error.message; }); }
|
||||||
|
function resetCredential() { state.credentialId = ''; element('#credential-name').value = ''; element('#api-key').value = ''; element('#service-account').value = ''; element('#api-key').placeholder = ''; element('#service-account').placeholder = ''; element('#rpm').value = 45; element('#save-credential').textContent = 'Add credential profile'; element('#cancel-credential').hidden = true; }
|
||||||
|
function resetDocument() { state.documentId = ''; state.discoveredSheets = []; element('#document-name').value = ''; element('#document-url').value = ''; element('#sheet-picker').innerHTML = ''; element('#save-document').textContent = 'Add document'; element('#cancel-document').hidden = true; }
|
||||||
|
function resetPublication() { state.publicationId = ''; element('#publication-name').value = ''; element('#cache').value = 30; element('#max-rows').value = 10000; element('#consumer-rpm').value = 120; element('#scheduled-refresh').checked = false; element('#data-format').value = 'rows'; element('#sheet-range').value = ''; element('#omit-blank-rows').checked = false; element('#publish').textContent = 'Publish endpoint'; element('#cancel-publication').hidden = true; renderEndpointPicker(); }
|
||||||
|
|
||||||
|
function bindRecordActions() {
|
||||||
|
document.querySelectorAll('[data-edit-credential]').forEach((button) => button.addEventListener('click', () => {
|
||||||
|
const credential = state.config.credentials.find((item) => item.id === button.dataset.editCredential); state.credentialId = credential.id; element('#credential-name').value = credential.name; element('#rpm').value = credential.requestsPerMinute; element(`input[name="mode"][value="${credential.mode}"]`).checked = true; applyCredentialMode(); element('#api-key').value = credential.mode === 'apiKey' ? secretMask : ''; element('#service-account').value = credential.mode === 'serviceAccount' ? secretMask : ''; element('#save-credential').textContent = 'Save credential changes'; element('#cancel-credential').hidden = false; showPage('credentials'); status('Credential loaded. Replace the dots only to change its saved secret.');
|
||||||
|
}));
|
||||||
|
document.querySelectorAll('[data-edit-document]').forEach((button) => button.addEventListener('click', () => {
|
||||||
|
const item = state.config.documents.find((document) => document.id === button.dataset.editDocument); state.documentId = item.id; state.discoveredSheets = item.sheets; element('#document-name').value = item.name; element('#document-url').value = `https://docs.google.com/spreadsheets/d/${item.spreadsheetId}/edit`; element('#document-credential').value = item.credentialId; renderPicker('#sheet-picker', item.sheets); element('#save-document').textContent = 'Save document changes'; element('#cancel-document').hidden = false; showPage('documents'); status('Document loaded. Load sheet tabs again to refresh the available names.');
|
||||||
|
}));
|
||||||
|
document.querySelectorAll('[data-edit-publication]').forEach((button) => button.addEventListener('click', () => {
|
||||||
|
const item = state.config.publications.find((publication) => publication.id === button.dataset.editPublication); state.publicationId = item.id; element('#publication-name').value = item.name; element('#publication-document').value = item.documentId; element('#cache').value = item.cacheSeconds; element('#max-rows').value = item.maxRows; element('#consumer-rpm').value = item.consumerRequestsPerMinute; element('#scheduled-refresh').checked = Boolean(item.scheduledRefresh); element('#data-format').value = item.dataFormat || 'rows'; element('#sheet-range').value = item.range || ''; element('#omit-blank-rows').checked = Boolean(item.omitBlankRows); renderEndpointPicker(item.sheets.map((sheet) => sheet.title)); element('#publish').textContent = 'Save endpoint changes'; element('#cancel-publication').hidden = false; showPage('endpoints'); status('Endpoint loaded.');
|
||||||
|
}));
|
||||||
|
document.querySelectorAll('[data-copy-token]').forEach((button) => button.addEventListener('click', async () => {
|
||||||
|
try {
|
||||||
|
const secret = await adminFetch(`/admin/publications/${button.dataset.copyToken}/token`);
|
||||||
|
await navigator.clipboard.writeText(secret.token);
|
||||||
|
button.textContent = 'Copied';
|
||||||
|
setTimeout(() => { button.textContent = 'Copy token'; }, 1_500);
|
||||||
|
} catch (error) { status(error.message, true); }
|
||||||
|
}));
|
||||||
|
document.querySelectorAll('[data-rotate-token]').forEach((button) => button.addEventListener('click', async () => {
|
||||||
|
if (!confirm('Rotate this token? Existing clients will stop working immediately.')) return;
|
||||||
|
try {
|
||||||
|
const secret = await adminFetch(`/admin/publications/${button.dataset.rotateToken}/rotate-token`, { method: 'POST' });
|
||||||
|
await navigator.clipboard.writeText(secret.token);
|
||||||
|
button.textContent = 'New token copied';
|
||||||
|
setTimeout(() => { button.textContent = 'Rotate token'; }, 1_500);
|
||||||
|
status('Token rotated and copied.');
|
||||||
|
} catch (error) { status(error.message, true); }
|
||||||
|
}));
|
||||||
|
document.querySelectorAll('[data-refresh-publication]').forEach((button) => button.addEventListener('click', async () => {
|
||||||
|
try { await adminFetch(`/admin/publications/${button.dataset.refreshPublication}/refresh`, { method: 'POST' }); button.textContent = 'Refreshed'; setTimeout(() => { button.textContent = 'Refresh'; }, 1_500); } catch (error) { status(error.message, true); }
|
||||||
|
}));
|
||||||
|
document.querySelectorAll('[data-toggle-publication]').forEach((button) => button.addEventListener('click', async () => {
|
||||||
|
try { const publication = state.config.publications.find((item) => item.id === button.dataset.togglePublication); await adminFetch(`/admin/publications/${publication.id}/enabled`, { method: 'POST', body: JSON.stringify({ enabled: !publication.enabled }) }); await loadConfig(); } catch (error) { status(error.message, true); }
|
||||||
|
}));
|
||||||
|
['credential', 'document', 'publication'].forEach((type) => document.querySelectorAll(`[data-delete-${type}]`).forEach((button) => button.addEventListener('click', async () => { if (!confirm(`Delete this ${type}? This cannot be undone.`)) return; try { await adminFetch(`/admin/${type === 'publication' ? 'publications' : `${type}s`}/${button.dataset[`delete${type[0].toUpperCase()}${type.slice(1)}`]}`, { method: 'DELETE' }); await loadConfig(); status(`${type[0].toUpperCase()}${type.slice(1)} removed.`); } catch (error) { status(error.message, true); } })));
|
||||||
|
}
|
||||||
|
|
||||||
|
element('#connect').addEventListener('click', async () => { state.token = element('#admin-token').value.trim(); try { await loadConfig(); sessionStorage.setItem('adminToken', state.token); element('#access-panel').hidden = true; element('#console').hidden = false; status('Connected.'); } catch (error) { status(error.message, true); } });
|
||||||
|
document.querySelectorAll('.tab').forEach((button) => button.addEventListener('click', () => showPage(button.dataset.page)));
|
||||||
|
applyTheme(localStorage.getItem('theme') === 'dark');
|
||||||
|
applyCredentialMode();
|
||||||
|
document.querySelectorAll('input[name="mode"]').forEach((input) => input.addEventListener('change', applyCredentialMode));
|
||||||
|
element('#dark-mode').addEventListener('change', (event) => {
|
||||||
|
const isDark = event.target.checked;
|
||||||
|
localStorage.setItem('theme', isDark ? 'dark' : 'light');
|
||||||
|
applyTheme(isDark);
|
||||||
|
});
|
||||||
|
element('#save-settings').addEventListener('click', async () => {
|
||||||
|
try {
|
||||||
|
const corsOrigins = element('#cors-origins').value.split(/\r?\n/);
|
||||||
|
await adminFetch('/admin/settings', { method: 'PUT', body: JSON.stringify({ corsOrigins, trustedProxyHops: element('#trusted-proxy-hops').value }) });
|
||||||
|
await loadConfig();
|
||||||
|
status('CORS settings saved.');
|
||||||
|
} catch (error) { status(error.message, true); }
|
||||||
|
});
|
||||||
|
element('#service-account-file').addEventListener('change', async (event) => { const [file] = event.target.files; if (file) element('#service-account').value = await file.text(); });
|
||||||
|
element('#save-credential').addEventListener('click', async () => { try { await adminFetch('/admin/credentials', { method: 'POST', body: JSON.stringify({ id: state.credentialId || undefined, name: element('#credential-name').value, mode: element('input[name="mode"]:checked').value, apiKey: element('#api-key').value === secretMask ? '' : element('#api-key').value, serviceAccountJson: element('#service-account').value === secretMask ? '' : element('#service-account').value, requestsPerMinute: element('#rpm').value }) }); await loadConfig(); resetCredential(); applyCredentialMode(); status('Credential saved.'); } catch (error) { status(error.message, true); } });
|
||||||
|
element('#cancel-credential').addEventListener('click', resetCredential);
|
||||||
|
element('#discover').addEventListener('click', async () => { try { const result = await adminFetch('/admin/documents/discover', { method: 'POST', body: JSON.stringify({ credentialId: element('#document-credential').value, url: element('#document-url').value }) }); state.discoveredSheets = result.sheets; renderPicker('#sheet-picker', result.sheets); status(`Loaded ${result.sheets.length} tab(s) from ${result.title}.`); } catch (error) { status(error.message, true); } });
|
||||||
|
element('#save-document').addEventListener('click', async () => { try { const sheets = state.discoveredSheets.filter((sheet) => selectedTitles('#sheet-picker').includes(sheet.title)); await adminFetch('/admin/documents', { method: 'POST', body: JSON.stringify({ id: state.documentId || undefined, name: element('#document-name').value, url: element('#document-url').value, credentialId: element('#document-credential').value, sheets }) }); await loadConfig(); resetDocument(); status('Document saved.'); } catch (error) { status(error.message, true); } });
|
||||||
|
element('#cancel-document').addEventListener('click', resetDocument);
|
||||||
|
element('#publication-document').addEventListener('change', () => renderEndpointPicker());
|
||||||
|
element('#publish').addEventListener('click', async () => { try { const selectedDocument = state.config.documents.find((item) => item.id === element('#publication-document').value); if (!selectedDocument) throw new Error('Select a document first.'); const titles = selectedTitles('#endpoint-sheet-picker'); const sheets = selectedDocument.sheets.filter((sheet) => titles.includes(sheet.title)); await adminFetch('/admin/publications', { method: 'POST', body: JSON.stringify({ id: state.publicationId || undefined, name: element('#publication-name').value, documentId: selectedDocument.id, sheets, cacheSeconds: element('#cache').value, maxRows: element('#max-rows').value, consumerRequestsPerMinute: element('#consumer-rpm').value, scheduledRefresh: element('#scheduled-refresh').checked, dataFormat: element('#data-format').value, range: element('#sheet-range').value, omitBlankRows: element('#omit-blank-rows').checked }) }); await loadConfig(); resetPublication(); status('Endpoint saved.'); } catch (error) { status(error.message, true); } });
|
||||||
|
['#request-endpoint', '#request-status', '#request-result'].forEach((selector) => element(selector).addEventListener('change', () => loadRequests().catch((error) => { element('#request-log').textContent = error.message; })));
|
||||||
|
element('#clear-request-log').addEventListener('click', async () => { if (!confirm('Clear all retained API request logs?')) return; try { await adminFetch('/admin/requests', { method: 'DELETE' }); await loadRequests(); } catch (error) { status(error.message, true); } });
|
||||||
|
element('#rotate-admin-token').addEventListener('click', async () => { if (!confirm('Rotate the admin token? Other admin sessions will stop working.')) return; try { const result = await adminFetch('/admin/token/rotate', { method: 'POST' }); state.token = result.token; sessionStorage.setItem('adminToken', state.token); await navigator.clipboard.writeText(result.token); status('Admin token rotated and copied.'); } catch (error) { status(error.message, true); } });
|
||||||
|
element('#export-config').addEventListener('click', async () => { try { const exported = await adminFetch('/admin/config/export'); const url = URL.createObjectURL(new Blob([JSON.stringify(exported, null, 2)], { type: 'application/json' })); const link = document.createElement('a'); link.href = url; link.download = 'sheet-relay-config.json'; link.click(); URL.revokeObjectURL(url); } catch (error) { status(error.message, true); } });
|
||||||
|
element('#import-config').addEventListener('click', () => element('#config-file').click());
|
||||||
|
element('#config-file').addEventListener('change', async (event) => { const [file] = event.target.files; if (!file) return; try { const imported = JSON.parse(await file.text()); if (!confirm('Replace all existing configuration with this import?')) return; await adminFetch('/admin/config/import', { method: 'POST', body: JSON.stringify(imported) }); await loadConfig(); await loadStatus(); status('Configuration imported. Add credential secrets, then enable endpoints.'); } catch (error) { status(error.message, true); } finally { event.target.value = ''; } });
|
||||||
|
element('#cancel-publication').addEventListener('click', resetPublication);
|
||||||
|
|
||||||
|
if (state.token) {
|
||||||
|
loadConfig().then(() => {
|
||||||
|
element('#admin-token').value = state.token;
|
||||||
|
element('#access-panel').hidden = true;
|
||||||
|
element('#console').hidden = false;
|
||||||
|
}).catch(() => {
|
||||||
|
sessionStorage.removeItem('adminToken');
|
||||||
|
state.token = '';
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>Sheet Relay</title>
|
||||||
|
<link rel="stylesheet" href="/styles.css">
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<main>
|
||||||
|
<header><div class="header-row"><div><p class="eyebrow">GOOGLE SHEETS TO API</p><h1>Sheet Relay</h1></div><label class="theme-toggle" title="Toggle dark mode"><input id="dark-mode" type="checkbox" aria-label="Toggle dark mode"><span class="theme-icon sun" aria-hidden="true"><i data-lucide="sun"></i></span><span class="theme-icon moon" aria-hidden="true"><i data-lucide="moon"></i></span></label></div><p>Choose exactly which spreadsheet tabs become authenticated JSON endpoints.</p></header>
|
||||||
|
<section id="access-panel" class="panel">
|
||||||
|
<h2>Admin access</h2><div class="inline"><input id="admin-token" type="password" placeholder="Admin token from server startup"><button id="connect" title="Connect to admin console">Connect</button></div><p id="status" class="status">Enter the token printed when the server first starts.</p>
|
||||||
|
</section>
|
||||||
|
<div id="console" hidden>
|
||||||
|
<nav class="tabs" aria-label="Configuration pages"><button class="tab active" data-page="credentials">Credentials</button><button class="tab" data-page="documents">Documents</button><button class="tab" data-page="endpoints">Endpoints</button><button class="tab" data-page="operations">Operations</button></nav>
|
||||||
|
<section class="page panel" id="credentials"><h2>Credential profiles</h2>
|
||||||
|
<div class="two-up"><label>Profile name<input id="credential-name" placeholder="Production service account"></label><label>Requests per minute<input id="rpm" type="number" min="1" max="10000" value="45"></label></div>
|
||||||
|
<div class="mode"><label><input type="radio" name="mode" value="apiKey" checked> API key</label><label><input type="radio" name="mode" value="serviceAccount"> Service account</label></div>
|
||||||
|
<label id="api-key-field">Google API key<input id="api-key" type="password" autocomplete="off"></label>
|
||||||
|
<label id="service-account-field">Service account JSON<textarea id="service-account" rows="5" spellcheck="false"></textarea></label>
|
||||||
|
<label id="service-account-upload" class="upload">Upload service-account JSON<input id="service-account-file" type="file" accept="application/json,.json"></label>
|
||||||
|
<button id="save-credential">Add credential profile</button><button id="cancel-credential" class="secondary" hidden>Cancel edit</button><p class="hint">45 RPM leaves room below a 50 RPM quota. API keys require publicly readable sheets. Share private sheets with the service account's client email.</p><div id="credentials-list" class="records"></div>
|
||||||
|
</section>
|
||||||
|
<section class="page panel" id="documents" hidden><h2>Google documents</h2>
|
||||||
|
<div class="two-up"><label>Document name<input id="document-name" placeholder="Warehouse inventory"></label><label>Credential profile<select id="document-credential"></select></label></div>
|
||||||
|
<label>Google Sheets URL<input id="document-url" type="url" placeholder="https://docs.google.com/spreadsheets/d/.../"></label>
|
||||||
|
<button id="discover">Load sheet tabs</button><div id="sheet-picker" class="sheet-picker"></div><button id="save-document">Add document</button><button id="cancel-document" class="secondary" hidden>Cancel edit</button><p class="hint">Paste the spreadsheet URL. Select tabs by name after they are loaded.</p><div id="documents-list" class="records"></div>
|
||||||
|
</section>
|
||||||
|
<section class="page panel" id="endpoints" hidden><h2>Authenticated endpoints</h2>
|
||||||
|
<div class="two-up"><label>Endpoint name<input id="publication-name" placeholder="Inventory feed"></label><label>Document<select id="publication-document"></select></label></div>
|
||||||
|
<label>Include sheet tabs<div id="endpoint-sheet-picker" class="sheet-picker"></div></label>
|
||||||
|
<div class="two-up"><label>Output format<select id="data-format"><option value="rows">Rows</option><option value="objects">Objects using header row</option></select></label><label>A1 range (optional)<input id="sheet-range" placeholder="A1:H500"></label></div>
|
||||||
|
<label class="check-option"><input id="omit-blank-rows" type="checkbox"> Omit blank rows</label>
|
||||||
|
<div class="two-up"><label>Cache seconds<input id="cache" type="number" min="0" max="86400" value="30"></label><label>Maximum rows per sheet<input id="max-rows" type="number" min="1" max="100000" value="10000"></label></div>
|
||||||
|
<label>Consumer requests per minute<input id="consumer-rpm" type="number" min="1" max="100000" value="120"></label>
|
||||||
|
<label class="check-option"><input id="scheduled-refresh" type="checkbox"> Background cache refresh</label>
|
||||||
|
<button id="publish">Publish endpoint</button><button id="cancel-publication" class="secondary" hidden>Cancel edit</button><p class="hint">A bearer token is generated automatically. Copy it or rotate it from the endpoint controls after publishing.</p><details class="response-format"><summary>Response format</summary><pre>{
|
||||||
|
"publication": "Inventory feed",
|
||||||
|
"document": "Warehouse inventory",
|
||||||
|
"spreadsheetId": "...",
|
||||||
|
"updatedAt": "2026-09-17T12:00:00.000Z",
|
||||||
|
"sheets": [{ "sheet": "OUTPUT", "values": [["Header"], ["Value"]] }],
|
||||||
|
"cached": false
|
||||||
|
}</pre></details><div id="publications" class="records"></div>
|
||||||
|
</section>
|
||||||
|
<section class="page panel" id="operations" hidden><h2>Operations</h2>
|
||||||
|
<label>Allowed browser origins<textarea id="cors-origins" rows="4" spellcheck="false" placeholder="* https://app.example.com"></textarea></label>
|
||||||
|
<label>Trusted reverse proxy hops<input id="trusted-proxy-hops" type="number" min="0" max="10" value="0"></label>
|
||||||
|
<button id="save-settings">Save CORS settings</button><p class="hint">Browser origins are denied by default. Use * for all origins, or enter one exact origin per line to allow selected browser clients. Server-to-server callers without an Origin header are unaffected.</p>
|
||||||
|
<p class="hint">Set proxy hops to 1 when one trusted reverse proxy is in front of this service. This makes request logs use the original client address from forwarded headers.</p>
|
||||||
|
<h3>Administration</h3><div class="actions operations-actions"><button id="rotate-admin-token" class="danger">Rotate admin token</button><button id="export-config" class="secondary">Export redacted config</button><button id="import-config" class="secondary">Import config</button><input id="config-file" type="file" accept="application/json,.json" hidden></div>
|
||||||
|
<p class="hint">Imports replace all existing configuration. Credentials are imported without their secrets, endpoints receive new tokens, and remain disabled until you explicitly enable them.</p>
|
||||||
|
<h3>Metrics</h3><div class="status-card"><code id="metrics-url"></code></div>
|
||||||
|
<h3>Service status</h3><div id="service-status" class="status-card">Loading status when this page opens.</div>
|
||||||
|
<h3>Endpoint health</h3><div id="endpoint-health" class="request-log"></div>
|
||||||
|
<h3>Recent API requests</h3><div class="request-filters"><select id="request-endpoint"><option value="">All endpoints</option></select><select id="request-status"><option value="">All statuses</option><option value="200">200</option><option value="401">401</option><option value="429">429</option><option value="502">502</option><option value="503">503</option></select><select id="request-result"><option value="">All results</option><option value="cached">Cached</option><option value="refresh">Refreshed</option><option value="stale">Stale</option></select><button id="clear-request-log" class="danger">Clear log</button></div><div id="request-log" class="request-log"></div>
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
</main>
|
||||||
|
<script src="/vendor/lucide/lucide.min.js"></script>
|
||||||
|
<script src="/app.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
@import url('https://fonts.googleapis.com/css2?family=DM+Mono:wght@400;500&family=Fraunces:opsz,wght@9..144,600;9..144,700&display=swap');
|
||||||
|
|
||||||
|
:root { --ink: #15231f; --muted: #60706a; --paper: #f6f3ea; --rule: #e8e5da; --panel: #fffdf7; --field: #fffefa; --line: #d8ddd2; --field-line: #abb8b0; --accent: #d8562b; --dark: #173c35; --button-ink: #fff; --shadow: #dfe5d8; --soft: #eaf0e7; --secondary: #b7c7bd; --secondary-hover: #91a99a; }
|
||||||
|
:root[data-theme="dark"] { --ink: #e9eee7; --muted: #b4c1b9; --paper: #17201d; --rule: #29332f; --panel: #202a26; --field: #18221e; --line: #405149; --field-line: #61766b; --accent: #ef855d; --dark: #9ec9b4; --button-ink: #15231f; --shadow: #0d1311; --soft: #293a33; --secondary: #596f64; --secondary-hover: #708a7b; }
|
||||||
|
* { box-sizing: border-box; }
|
||||||
|
body { margin: 0; color: var(--ink); background: repeating-linear-gradient(90deg, transparent 0 39px, var(--rule) 40px), var(--paper); font-family: "DM Mono", monospace; font-size: 14px; }
|
||||||
|
main { width: min(880px, calc(100% - 32px)); margin: 0 auto; padding: 64px 0; }
|
||||||
|
header { border-left: 8px solid var(--accent); padding-left: 20px; margin-bottom: 36px; }.header-row { display: flex; align-items: start; justify-content: space-between; gap: 16px; }
|
||||||
|
.eyebrow, .hint { color: var(--muted); font-size: 12px; line-height: 1.6; }
|
||||||
|
h1, h2 { font-family: Fraunces, serif; letter-spacing: 0; } h1 { font-size: 48px; margin: 3px 0 10px; } h2 { margin: 0 0 20px; font-size: 25px; }
|
||||||
|
.panel { background: var(--panel); border: 1px solid var(--line); border-radius: 6px; padding: 28px; margin: 18px 0; box-shadow: 5px 5px 0 var(--shadow); }
|
||||||
|
label { display: grid; gap: 8px; margin: 14px 0; } [hidden] { display: none !important; } input, select, textarea { width: 100%; min-height: 42px; border: 1px solid var(--field-line); border-radius: 3px; background: var(--field); padding: 10px 11px; font: inherit; color: var(--ink); } select { appearance: auto; } textarea { resize: vertical; }
|
||||||
|
button { border: 0; border-radius: 3px; background: var(--dark); color: var(--button-ink); font: 500 13px "DM Mono", monospace; padding: 11px 16px; cursor: pointer; } button:hover { background: var(--accent); color: #fff; } button.danger { background: #a83129; color: #fff; } button.secondary { background: var(--secondary); color: var(--ink); } button.secondary:hover { background: var(--secondary-hover); }.secret-input { display: flex; gap: 8px; }.secret-input input { min-width: 0; }.secret-input button { flex: 0 0 auto; }.icon-button { padding: 8px 10px; }
|
||||||
|
.panel > button + button { margin-left: 8px; }
|
||||||
|
.tabs { display: flex; gap: 1px; margin: 28px 0 -18px; }.tab { border-radius: 4px 4px 0 0; background: var(--secondary); color: var(--ink); }.tab.active { background: var(--dark); color: var(--button-ink); }#console .page { border-radius: 0 6px 6px 6px; }
|
||||||
|
.inline, .two-up, .mode { display: flex; gap: 14px; align-items: end; } .two-up > label { flex: 1; } .mode { align-items: center; } .mode label { display: flex; margin: 0; align-items: center; } .mode input { width: auto; }
|
||||||
|
.status { min-height: 20px; color: var(--accent); } .sheet-picker { display: grid; grid-template-columns: repeat(auto-fit, minmax(145px, 1fr)); gap: 8px; margin: 18px 0; }.sheet-picker label { display: flex; align-items: center; border: 1px solid var(--line); padding: 10px; margin: 0; gap: 8px; }.sheet-picker input { width: auto; }
|
||||||
|
.check-option { display: flex; align-items: center; gap: 8px; }.check-option input { width: auto; min-height: auto; }
|
||||||
|
input[type="checkbox"] { appearance: none; display: grid; place-content: center; width: 18px; min-width: 18px; height: 18px; min-height: 18px; margin: 0; padding: 0; border: 1px solid var(--field-line); border-radius: 3px; background: var(--field); cursor: pointer; } input[type="checkbox"]::after { content: ""; width: 7px; height: 4px; border: solid var(--button-ink); border-width: 0 0 2px 2px; transform: rotate(-45deg) scale(0); transform-origin: center; } input[type="checkbox"]:checked { border-color: var(--dark); background: var(--dark); } input[type="checkbox"]:checked::after { transform: rotate(-45deg) scale(1); }
|
||||||
|
.records { margin-top: 24px; }.record, .endpoint { border-top: 1px solid var(--line); padding: 16px 0; display: flex; justify-content: space-between; gap: 14px; align-items: start; }.record:first-child, .endpoint:first-child { border-top: 0; padding-top: 0; }.endpoint { flex-wrap: wrap; }.endpoint code { display: inline-block; max-width: 100%; color: var(--muted); overflow-wrap: anywhere; font-size: 11px; margin: 5px 0; }.endpoint pre { width: 100%; max-width: 620px; overflow: auto; white-space: pre-wrap; background: var(--soft); border-left: 3px solid var(--accent); padding: 12px; font-size: 11px; line-height: 1.5; }.actions { display: flex; flex-wrap: wrap; justify-content: flex-end; gap: 6px; }.actions button { padding: 8px 10px; }
|
||||||
|
.response-format { margin: 18px 0; color: var(--muted); }.response-format summary { cursor: pointer; }.response-format pre { max-width: 620px; overflow: auto; background: var(--soft); border-left: 3px solid var(--accent); padding: 12px; color: var(--ink); font-size: 11px; line-height: 1.5; }
|
||||||
|
.status-card { border-left: 3px solid var(--dark); background: var(--soft); padding: 12px; line-height: 1.5; }.request-log { margin-top: 12px; }.log-entry { display: grid; grid-template-columns: 52px minmax(0, 1fr) 90px 64px 170px; gap: 10px; align-items: center; border-top: 1px solid var(--line); padding: 10px 0; font-size: 11px; }.log-entry strong { color: var(--accent); }.log-entry span { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }.log-entry time { color: var(--muted); }.theme-toggle { display: grid; position: relative; width: 32px; height: 32px; place-items: center; margin: 4px 0; border: 1px solid var(--field-line); border-radius: 50%; background: var(--field); cursor: pointer; }.theme-toggle input { position: absolute; opacity: 0; inset: 0; margin: 0; cursor: pointer; }.theme-icon { display: grid; place-items: center; color: var(--accent); }.theme-icon svg { width: 18px; height: 18px; stroke-width: 2; }.theme-icon.moon { display: none; }:root[data-theme="dark"] .theme-icon.sun { display: none; }:root[data-theme="dark"] .theme-icon.moon { display: grid; }
|
||||||
|
.health-entry { display: grid; grid-template-columns: minmax(120px, 1fr) 68px minmax(170px, 1.5fr); gap: 10px; align-items: center; border-top: 1px solid var(--line); padding: 10px 0; font-size: 11px; }.health-entry span { color: var(--muted); }.health-entry .health-error { color: #a83129; grid-column: 1 / -1; overflow-wrap: anywhere; }
|
||||||
|
.request-filters, .operations-actions { display: flex; flex-wrap: wrap; gap: 8px; align-items: end; margin: 12px 0; }.request-filters select { width: auto; min-width: 130px; min-height: 36px; padding: 6px 8px; }.request-filters button { padding: 9px 12px; }.operations-actions { justify-content: flex-start; }
|
||||||
|
@media (max-width: 600px) { .log-entry { grid-template-columns: 45px minmax(0, 1fr); }.log-entry time { grid-column: 1 / -1; } }
|
||||||
|
@media (max-width: 600px) { main { padding: 34px 0; } h1 { font-size: 38px; }.panel { padding: 20px; }.two-up, .inline { display: grid; }.inline button { width: 100%; } }
|
||||||
@@ -0,0 +1,510 @@
|
|||||||
|
import { createHash, randomBytes, randomUUID, timingSafeEqual } from 'node:crypto';
|
||||||
|
import express from 'express';
|
||||||
|
import { loadConfig, publicConfig, saveConfig } from './lib/config-store.js';
|
||||||
|
import { RateLimiter, createSheetsClient, listTabs, readTabs } from './lib/google-sheets.js';
|
||||||
|
import { clearRequests, recentRequests, recordRequest } from './lib/request-log.js';
|
||||||
|
|
||||||
|
const app = express();
|
||||||
|
const config = await loadConfig();
|
||||||
|
const limiters = new Map();
|
||||||
|
const consumerLimiters = new Map();
|
||||||
|
const cache = new Map();
|
||||||
|
const endpointHealth = new Map();
|
||||||
|
const refreshingPublications = new Map();
|
||||||
|
const tabCache = new Map();
|
||||||
|
const tabCacheMilliseconds = 5 * 60 * 1000;
|
||||||
|
const refreshIntervalMilliseconds = 5_000;
|
||||||
|
const startedAt = Date.now();
|
||||||
|
const metrics = { requests: 0, errors: 0, staleResponses: 0, refreshes: 0, refreshErrors: 0, refreshDurationMilliseconds: 0 };
|
||||||
|
|
||||||
|
app.set('trust proxy', config.settings.trustedProxyHops);
|
||||||
|
|
||||||
|
app.use(express.json({ limit: '1mb' }));
|
||||||
|
app.use('/api', (request, response, next) => {
|
||||||
|
const origin = request.get('origin');
|
||||||
|
if (!origin) return next();
|
||||||
|
if (!config.settings.corsOrigins.includes('*') && !config.settings.corsOrigins.includes(origin)) return response.status(403).json({ error: 'Origin is not permitted.' });
|
||||||
|
response.set('Access-Control-Allow-Origin', origin);
|
||||||
|
response.set('Vary', 'Origin');
|
||||||
|
response.set('Access-Control-Allow-Headers', 'Authorization, Content-Type');
|
||||||
|
response.set('Access-Control-Allow-Methods', 'GET, OPTIONS');
|
||||||
|
if (request.method === 'OPTIONS') return response.status(204).end();
|
||||||
|
next();
|
||||||
|
});
|
||||||
|
app.use((request, response, next) => {
|
||||||
|
request.startedAt = Date.now();
|
||||||
|
next();
|
||||||
|
});
|
||||||
|
app.use('/vendor/lucide', express.static('node_modules/lucide/dist/umd'));
|
||||||
|
app.use(express.static('public'));
|
||||||
|
|
||||||
|
function sameSecret(received, expected) {
|
||||||
|
if (!received || !expected) return false;
|
||||||
|
const receivedBuffer = Buffer.from(received);
|
||||||
|
const expectedBuffer = Buffer.from(expected);
|
||||||
|
return receivedBuffer.length === expectedBuffer.length && timingSafeEqual(receivedBuffer, expectedBuffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
function requireAdmin(request, response, next) {
|
||||||
|
if (!sameSecret(request.get('x-admin-token'), config.adminToken)) {
|
||||||
|
return response.status(401).json({ error: 'Admin authentication required.' });
|
||||||
|
}
|
||||||
|
next();
|
||||||
|
}
|
||||||
|
|
||||||
|
function requiredText(value, label) {
|
||||||
|
if (!value?.trim()) throw new Error(`${label} is required.`);
|
||||||
|
return value.trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
function spreadsheetIdFrom(value) {
|
||||||
|
const input = requiredText(value, 'Google Sheets URL');
|
||||||
|
const match = input.match(/\/spreadsheets\/d\/([a-zA-Z0-9_-]+)/);
|
||||||
|
if (match) return match[1];
|
||||||
|
if (/^[a-zA-Z0-9_-]{20,}$/.test(input)) return input;
|
||||||
|
throw new Error('Enter a valid Google Sheets URL.');
|
||||||
|
}
|
||||||
|
|
||||||
|
function credentialFor(id) {
|
||||||
|
const credential = config.credentials.find((item) => item.id === id);
|
||||||
|
if (!credential) throw new Error('The selected credential profile no longer exists.');
|
||||||
|
return credential;
|
||||||
|
}
|
||||||
|
|
||||||
|
function documentFor(id) {
|
||||||
|
const document = config.documents.find((item) => item.id === id);
|
||||||
|
if (!document) throw new Error('The selected document no longer exists.');
|
||||||
|
return document;
|
||||||
|
}
|
||||||
|
|
||||||
|
function limiterFor(credential) {
|
||||||
|
const identity = credential.mode === 'serviceAccount'
|
||||||
|
? JSON.parse(credential.serviceAccountJson).client_email
|
||||||
|
: credential.apiKey;
|
||||||
|
const key = createHash('sha256').update(`${credential.mode}:${identity}`).digest('hex');
|
||||||
|
if (!limiters.has(key)) limiters.set(key, new RateLimiter());
|
||||||
|
return limiters.get(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
function consumerIsLimited(publication) {
|
||||||
|
const now = Date.now();
|
||||||
|
const windowMilliseconds = 60_000;
|
||||||
|
const limit = Math.max(1, Number(publication.consumerRequestsPerMinute) || 120);
|
||||||
|
const current = consumerLimiters.get(publication.id) || { startedAt: now, count: 0 };
|
||||||
|
if (now - current.startedAt >= windowMilliseconds) {
|
||||||
|
current.startedAt = now;
|
||||||
|
current.count = 0;
|
||||||
|
}
|
||||||
|
if (current.count >= limit) return true;
|
||||||
|
current.count += 1;
|
||||||
|
consumerLimiters.set(publication.id, current);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshPublication(publication) {
|
||||||
|
if (refreshingPublications.has(publication.id)) return refreshingPublications.get(publication.id);
|
||||||
|
const refresh = (async () => {
|
||||||
|
const health = endpointHealth.get(publication.id) || {};
|
||||||
|
health.lastAttemptAt = new Date().toISOString();
|
||||||
|
try {
|
||||||
|
const refreshStartedAt = Date.now();
|
||||||
|
const document = documentFor(publication.documentId);
|
||||||
|
const credential = credentialFor(document.credentialId);
|
||||||
|
const payload = {
|
||||||
|
publication: publication.name,
|
||||||
|
document: document.name,
|
||||||
|
spreadsheetId: document.spreadsheetId,
|
||||||
|
updatedAt: new Date().toISOString(),
|
||||||
|
sheets: await readTabs(createSheetsClient(credential), {
|
||||||
|
spreadsheetId: document.spreadsheetId,
|
||||||
|
sheets: publication.sheets,
|
||||||
|
dataFormat: publication.dataFormat,
|
||||||
|
range: publication.range,
|
||||||
|
omitBlankRows: publication.omitBlankRows
|
||||||
|
}, limiterFor(credential), credential.requestsPerMinute)
|
||||||
|
};
|
||||||
|
cache.set(publication.id, { payload, expiresAt: Date.now() + publication.cacheSeconds * 1000 });
|
||||||
|
endpointHealth.set(publication.id, { ...health, lastSuccessAt: new Date().toISOString(), lastError: '' });
|
||||||
|
metrics.refreshes += 1;
|
||||||
|
metrics.refreshDurationMilliseconds += Date.now() - refreshStartedAt;
|
||||||
|
return payload;
|
||||||
|
} catch (error) {
|
||||||
|
endpointHealth.set(publication.id, { ...health, lastError: error.message });
|
||||||
|
metrics.refreshErrors += 1;
|
||||||
|
throw error;
|
||||||
|
} finally {
|
||||||
|
refreshingPublications.delete(publication.id);
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
refreshingPublications.set(publication.id, refresh);
|
||||||
|
return refresh;
|
||||||
|
}
|
||||||
|
|
||||||
|
function endpointStatus(publication) {
|
||||||
|
const cached = cache.get(publication.id);
|
||||||
|
const health = endpointHealth.get(publication.id) || {};
|
||||||
|
return {
|
||||||
|
id: publication.id,
|
||||||
|
name: publication.name,
|
||||||
|
enabled: publication.enabled !== false,
|
||||||
|
cacheState: !cached ? 'empty' : cached.expiresAt > Date.now() ? 'fresh' : 'stale',
|
||||||
|
cacheExpiresAt: cached ? new Date(cached.expiresAt).toISOString() : null,
|
||||||
|
...health
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function tabsFor(credential, spreadsheetId) {
|
||||||
|
const identity = credential.mode === 'serviceAccount'
|
||||||
|
? JSON.parse(credential.serviceAccountJson).client_email
|
||||||
|
: credential.apiKey;
|
||||||
|
const key = createHash('sha256').update(`${credential.mode}:${identity}:${spreadsheetId}`).digest('hex');
|
||||||
|
const existing = tabCache.get(key);
|
||||||
|
if (existing?.expiresAt > Date.now()) return existing.result;
|
||||||
|
|
||||||
|
const result = listTabs(createSheetsClient(credential), spreadsheetId, limiterFor(credential), credential.requestsPerMinute);
|
||||||
|
tabCache.set(key, { expiresAt: Date.now() + tabCacheMilliseconds, result });
|
||||||
|
try {
|
||||||
|
return await result;
|
||||||
|
} catch (error) {
|
||||||
|
tabCache.delete(key);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function auditEndpointRequest(request, publicationId, status, details = {}) {
|
||||||
|
metrics.requests += 1;
|
||||||
|
if (status >= 400) metrics.errors += 1;
|
||||||
|
if (details.stale) metrics.staleResponses += 1;
|
||||||
|
recordRequest({
|
||||||
|
at: new Date().toISOString(),
|
||||||
|
publicationId,
|
||||||
|
status,
|
||||||
|
durationMs: Date.now() - request.startedAt,
|
||||||
|
remoteAddress: request.ip,
|
||||||
|
...details
|
||||||
|
}).catch((error) => console.error('Unable to write request log:', error));
|
||||||
|
}
|
||||||
|
|
||||||
|
app.get('/health', (request, response) => response.json({
|
||||||
|
status: 'ok',
|
||||||
|
uptimeSeconds: Math.floor((Date.now() - startedAt) / 1000),
|
||||||
|
publications: config.publications.length
|
||||||
|
}));
|
||||||
|
|
||||||
|
app.get('/metrics', requireAdmin, (request, response) => {
|
||||||
|
const averageRefreshSeconds = metrics.refreshes ? metrics.refreshDurationMilliseconds / metrics.refreshes / 1_000 : 0;
|
||||||
|
response.type('text/plain').send([
|
||||||
|
'# TYPE sheet_relay_requests_total counter',
|
||||||
|
`sheet_relay_requests_total ${metrics.requests}`,
|
||||||
|
'# TYPE sheet_relay_errors_total counter',
|
||||||
|
`sheet_relay_errors_total ${metrics.errors}`,
|
||||||
|
'# TYPE sheet_relay_stale_responses_total counter',
|
||||||
|
`sheet_relay_stale_responses_total ${metrics.staleResponses}`,
|
||||||
|
'# TYPE sheet_relay_refreshes_total counter',
|
||||||
|
`sheet_relay_refreshes_total ${metrics.refreshes}`,
|
||||||
|
'# TYPE sheet_relay_refresh_errors_total counter',
|
||||||
|
`sheet_relay_refresh_errors_total ${metrics.refreshErrors}`,
|
||||||
|
'# TYPE sheet_relay_refresh_duration_seconds gauge',
|
||||||
|
`sheet_relay_refresh_duration_seconds ${averageRefreshSeconds}`
|
||||||
|
].join('\n') + '\n');
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/admin/config', requireAdmin, (request, response) => response.json(publicConfig(config)));
|
||||||
|
|
||||||
|
app.post('/admin/token/rotate', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
config.adminToken = randomBytes(24).toString('base64url');
|
||||||
|
await saveConfig(config);
|
||||||
|
response.json({ token: config.adminToken });
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.put('/admin/settings', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const corsOrigins = Array.isArray(request.body.corsOrigins)
|
||||||
|
? request.body.corsOrigins.map((origin) => origin.trim()).filter(Boolean)
|
||||||
|
: [];
|
||||||
|
const trustedProxyHops = Math.max(0, Math.min(10, Number(request.body.trustedProxyHops) || 0));
|
||||||
|
if (corsOrigins.some((origin) => origin !== '*' && !/^https?:\/\/[^/]+$/i.test(origin))) throw new Error('CORS origins must be * or an http or https origin without a path.');
|
||||||
|
config.settings.corsOrigins = [...new Set(corsOrigins)];
|
||||||
|
config.settings.trustedProxyHops = trustedProxyHops;
|
||||||
|
app.set('trust proxy', trustedProxyHops);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.json(config.settings);
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/admin/status', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
response.json({
|
||||||
|
status: 'ok',
|
||||||
|
uptimeSeconds: Math.floor((Date.now() - startedAt) / 1000),
|
||||||
|
caches: { endpointEntries: cache.size, tabEntries: tabCache.size },
|
||||||
|
endpoints: config.publications.map(endpointStatus),
|
||||||
|
recentRequests: await recentRequests()
|
||||||
|
});
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/admin/requests', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const requests = await recentRequests(500);
|
||||||
|
const filtered = requests.filter((entry) => {
|
||||||
|
if (request.query.publicationId && entry.publicationId !== request.query.publicationId) return false;
|
||||||
|
if (request.query.status && String(entry.status) !== String(request.query.status)) return false;
|
||||||
|
if (request.query.result === 'cached' && !entry.cached) return false;
|
||||||
|
if (request.query.result === 'refresh' && entry.cached) return false;
|
||||||
|
if (request.query.result === 'stale' && !entry.stale) return false;
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
response.json(filtered);
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/admin/requests', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
await clearRequests();
|
||||||
|
response.status(204).end();
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/admin/config/export', requireAdmin, (request, response) => {
|
||||||
|
const exported = publicConfig(config);
|
||||||
|
response.json({
|
||||||
|
version: 1,
|
||||||
|
settings: exported.settings,
|
||||||
|
credentials: exported.credentials.map(({ configured, ...credential }) => credential),
|
||||||
|
documents: exported.documents,
|
||||||
|
publications: exported.publications.map(({ tokenConfigured, ...publication }) => ({ ...publication, enabled: false }))
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/config/import', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const imported = request.body;
|
||||||
|
if (!imported || !Array.isArray(imported.credentials) || !Array.isArray(imported.documents) || !Array.isArray(imported.publications)) throw new Error('Choose a valid exported configuration file.');
|
||||||
|
const credentialIds = new Map();
|
||||||
|
const credentials = imported.credentials.map((credential) => {
|
||||||
|
if (!['apiKey', 'serviceAccount'].includes(credential.mode)) throw new Error('Imported credential has an invalid authentication type.');
|
||||||
|
const id = randomUUID();
|
||||||
|
credentialIds.set(credential.id, id);
|
||||||
|
return { id, name: requiredText(credential.name, 'Imported credential name'), mode: credential.mode, apiKey: '', serviceAccountJson: '', requestsPerMinute: Math.max(1, Math.min(10_000, Number(credential.requestsPerMinute) || 45)) };
|
||||||
|
});
|
||||||
|
const documentIds = new Map();
|
||||||
|
const documents = imported.documents.map((document) => {
|
||||||
|
const credentialId = credentialIds.get(document.credentialId);
|
||||||
|
if (!credentialId || !Array.isArray(document.sheets) || !document.sheets.length) throw new Error('Imported documents must reference an included credential and have at least one sheet.');
|
||||||
|
const id = randomUUID();
|
||||||
|
documentIds.set(document.id, id);
|
||||||
|
return { id, name: requiredText(document.name, 'Imported document name'), spreadsheetId: requiredText(document.spreadsheetId, 'Imported spreadsheet ID'), credentialId, sheets: document.sheets };
|
||||||
|
});
|
||||||
|
const publications = imported.publications.map((publication) => {
|
||||||
|
const documentId = documentIds.get(publication.documentId);
|
||||||
|
if (!documentId || !Array.isArray(publication.sheets) || !publication.sheets.length) throw new Error('Imported endpoints must reference an included document and have at least one sheet.');
|
||||||
|
return { id: randomUUID(), name: requiredText(publication.name, 'Imported endpoint name'), documentId, sheets: publication.sheets, apiToken: randomBytes(24).toString('base64url'), cacheSeconds: Math.max(0, Math.min(86_400, Number(publication.cacheSeconds) || 0)), dataFormat: publication.dataFormat === 'objects' ? 'objects' : 'rows', range: String(publication.range || '').trim(), omitBlankRows: Boolean(publication.omitBlankRows), maxRows: Math.max(1, Math.min(100_000, Number(publication.maxRows) || 10_000)), consumerRequestsPerMinute: Math.max(1, Math.min(100_000, Number(publication.consumerRequestsPerMinute) || 120)), scheduledRefresh: Boolean(publication.scheduledRefresh), enabled: false };
|
||||||
|
});
|
||||||
|
config.settings = { corsOrigins: Array.isArray(imported.settings?.corsOrigins) ? imported.settings.corsOrigins : [], trustedProxyHops: Math.max(0, Math.min(10, Number(imported.settings?.trustedProxyHops) || 0)) };
|
||||||
|
config.credentials = credentials;
|
||||||
|
config.documents = documents;
|
||||||
|
config.publications = publications;
|
||||||
|
cache.clear();
|
||||||
|
endpointHealth.clear();
|
||||||
|
app.set('trust proxy', config.settings.trustedProxyHops);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.json(publicConfig(config));
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/credentials', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const { id, name, mode, apiKey, serviceAccountJson, requestsPerMinute } = request.body;
|
||||||
|
if (!['apiKey', 'serviceAccount'].includes(mode)) throw new Error('Choose an authentication type.');
|
||||||
|
const existing = config.credentials.find((item) => item.id === id);
|
||||||
|
const suppliedSecret = mode === 'apiKey' ? apiKey?.trim() : serviceAccountJson?.trim();
|
||||||
|
const secret = suppliedSecret || (existing?.mode === mode ? (mode === 'apiKey' ? existing.apiKey : existing.serviceAccountJson) : '');
|
||||||
|
requiredText(secret, mode === 'apiKey' ? 'Google API key' : 'Service account JSON');
|
||||||
|
if (mode === 'serviceAccount') JSON.parse(secret);
|
||||||
|
const credential = {
|
||||||
|
id: id || randomUUID(),
|
||||||
|
name: requiredText(name, 'Profile name'),
|
||||||
|
mode,
|
||||||
|
apiKey: mode === 'apiKey' ? secret : '',
|
||||||
|
serviceAccountJson: mode === 'serviceAccount' ? secret : '',
|
||||||
|
requestsPerMinute: Math.max(1, Math.min(10_000, Number(requestsPerMinute) || 45))
|
||||||
|
};
|
||||||
|
const index = config.credentials.findIndex((item) => item.id === credential.id);
|
||||||
|
if (index >= 0) config.credentials[index] = credential;
|
||||||
|
else config.credentials.push(credential);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.status(201).json(publicConfig(config).credentials);
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/admin/credentials/:id', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
if (config.documents.some((document) => document.credentialId === request.params.id)) throw new Error('Remove or reassign documents using this profile first.');
|
||||||
|
config.credentials = config.credentials.filter((credential) => credential.id !== request.params.id);
|
||||||
|
limiters.delete(request.params.id);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.status(204).end();
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/documents/discover', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const credential = credentialFor(request.body.credentialId);
|
||||||
|
const spreadsheetId = spreadsheetIdFrom(request.body.url);
|
||||||
|
const result = await tabsFor(credential, spreadsheetId);
|
||||||
|
response.json({ ...result, spreadsheetId });
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/documents', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const { id, name, url, credentialId, sheets } = request.body;
|
||||||
|
credentialFor(credentialId);
|
||||||
|
if (!Array.isArray(sheets) || !sheets.length) throw new Error('Select at least one sheet tab.');
|
||||||
|
const document = { id: id || randomUUID(), name: requiredText(name, 'Document name'), spreadsheetId: spreadsheetIdFrom(url), credentialId, sheets };
|
||||||
|
const index = config.documents.findIndex((item) => item.id === document.id);
|
||||||
|
if (index >= 0) config.documents[index] = document;
|
||||||
|
else config.documents.push(document);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.status(201).json(publicConfig(config).documents);
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/admin/documents/:id', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
if (config.publications.some((publication) => publication.documentId === request.params.id)) throw new Error('Remove endpoints using this document first.');
|
||||||
|
config.documents = config.documents.filter((document) => document.id !== request.params.id);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.status(204).end();
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/publications', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const { id, name, documentId, sheets, cacheSeconds, dataFormat, range, omitBlankRows, maxRows, consumerRequestsPerMinute, scheduledRefresh } = request.body;
|
||||||
|
const document = documentFor(documentId);
|
||||||
|
const existing = config.publications.find((item) => item.id === id);
|
||||||
|
const selectedSheets = Array.isArray(sheets) && sheets.length ? sheets : document.sheets;
|
||||||
|
if (!selectedSheets.every((sheet) => document.sheets.some((available) => available.title === sheet.title))) throw new Error('Endpoint sheets must belong to the selected document.');
|
||||||
|
if (!['rows', 'objects'].includes(dataFormat || 'rows')) throw new Error('Choose a valid data format.');
|
||||||
|
const publication = { id: id || randomUUID(), name: requiredText(name, 'Endpoint name'), documentId, sheets: selectedSheets, apiToken: existing?.apiToken || randomBytes(24).toString('base64url'), cacheSeconds: Math.max(0, Math.min(86_400, Number(cacheSeconds) || 0)), dataFormat: dataFormat || 'rows', range: String(range || '').trim(), omitBlankRows: Boolean(omitBlankRows), maxRows: Math.max(1, Math.min(100_000, Number(maxRows) || 10_000)), consumerRequestsPerMinute: Math.max(1, Math.min(100_000, Number(consumerRequestsPerMinute) || 120)), scheduledRefresh: Boolean(scheduledRefresh), enabled: existing?.enabled !== false };
|
||||||
|
const index = config.publications.findIndex((item) => item.id === publication.id);
|
||||||
|
if (index >= 0) config.publications[index] = publication;
|
||||||
|
else config.publications.push(publication);
|
||||||
|
cache.delete(publication.id);
|
||||||
|
endpointHealth.delete(publication.id);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.status(201).json(publicConfig(config).publications);
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/admin/publications/:id', requireAdmin, async (request, response) => {
|
||||||
|
config.publications = config.publications.filter((publication) => publication.id !== request.params.id);
|
||||||
|
cache.delete(request.params.id);
|
||||||
|
consumerLimiters.delete(request.params.id);
|
||||||
|
endpointHealth.delete(request.params.id);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.status(204).end();
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/publications/:id/rotate-token', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const publication = config.publications.find((item) => item.id === request.params.id);
|
||||||
|
if (!publication) throw new Error('The endpoint no longer exists.');
|
||||||
|
publication.apiToken = randomBytes(24).toString('base64url');
|
||||||
|
cache.delete(publication.id);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.json({ token: publication.apiToken });
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/publications/:id/refresh', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const publication = config.publications.find((item) => item.id === request.params.id);
|
||||||
|
if (!publication) throw new Error('The endpoint no longer exists.');
|
||||||
|
await refreshPublication(publication);
|
||||||
|
response.json(endpointStatus(publication));
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/admin/publications/:id/enabled', requireAdmin, async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const publication = config.publications.find((item) => item.id === request.params.id);
|
||||||
|
if (!publication) throw new Error('The endpoint no longer exists.');
|
||||||
|
publication.enabled = Boolean(request.body.enabled);
|
||||||
|
await saveConfig(config);
|
||||||
|
response.json({ enabled: publication.enabled });
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/v1/:publicationId', async (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const publication = config.publications.find((item) => item.id === request.params.publicationId);
|
||||||
|
const token = request.get('authorization')?.replace(/^Bearer\s+/i, '');
|
||||||
|
if (!publication || !sameSecret(token, publication.apiToken)) {
|
||||||
|
auditEndpointRequest(request, request.params.publicationId, 401);
|
||||||
|
return response.status(401).json({ error: 'Invalid API credentials.' });
|
||||||
|
}
|
||||||
|
if (publication.enabled === false) {
|
||||||
|
auditEndpointRequest(request, publication.id, 503, { error: 'Endpoint is disabled.' });
|
||||||
|
return response.status(503).json({ error: 'This endpoint is temporarily disabled.' });
|
||||||
|
}
|
||||||
|
if (consumerIsLimited(publication)) {
|
||||||
|
auditEndpointRequest(request, publication.id, 429, { error: 'Consumer rate limit exceeded.' });
|
||||||
|
response.set('Retry-After', '60');
|
||||||
|
return response.status(429).json({ error: 'Consumer rate limit exceeded. Try again in a minute.' });
|
||||||
|
}
|
||||||
|
const cached = cache.get(publication.id);
|
||||||
|
if (cached && cached.expiresAt > Date.now()) {
|
||||||
|
auditEndpointRequest(request, publication.id, 200, { cached: true, stale: false });
|
||||||
|
return response.json({ ...cached.payload, cached: true, stale: false });
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const payload = await refreshPublication(publication);
|
||||||
|
auditEndpointRequest(request, publication.id, 200, { cached: false, stale: false });
|
||||||
|
response.json({ ...payload, cached: false, stale: false });
|
||||||
|
} catch (error) {
|
||||||
|
if (cached) {
|
||||||
|
auditEndpointRequest(request, publication.id, 200, { cached: true, stale: true, error: error.message });
|
||||||
|
return response.json({ ...cached.payload, cached: true, stale: true, staleAt: new Date(cached.expiresAt).toISOString() });
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
auditEndpointRequest(request, request.params.publicationId, 502, { error: error.message });
|
||||||
|
response.status(502).json({ error: error.message || 'Unable to refresh endpoint data.' });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.use((error, request, response, next) => {
|
||||||
|
console.error(error);
|
||||||
|
response.status(400).json({ error: error.message || 'Request failed.' });
|
||||||
|
});
|
||||||
|
|
||||||
|
const port = Number(process.env.PORT) || 3000;
|
||||||
|
app.listen(port, () => {
|
||||||
|
console.log(`Google Sheets API publisher listening on http://localhost:${port}`);
|
||||||
|
console.log('Admin token: data/config.json -> adminToken');
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/admin/publications/:id/token', requireAdmin, (request, response, next) => {
|
||||||
|
try {
|
||||||
|
const publication = config.publications.find((item) => item.id === request.params.id);
|
||||||
|
if (!publication) throw new Error('The endpoint no longer exists.');
|
||||||
|
response.json({ token: publication.apiToken });
|
||||||
|
} catch (error) { next(error); }
|
||||||
|
});
|
||||||
|
|
||||||
|
setInterval(() => {
|
||||||
|
for (const publication of config.publications) {
|
||||||
|
if (!publication.scheduledRefresh || publication.cacheSeconds <= 0) continue;
|
||||||
|
const cached = cache.get(publication.id);
|
||||||
|
const refreshLeadMilliseconds = Math.min(10_000, publication.cacheSeconds * 200);
|
||||||
|
if (!cached || cached.expiresAt - Date.now() <= refreshLeadMilliseconds) {
|
||||||
|
refreshPublication(publication).catch((error) => console.error(`Scheduled refresh failed for ${publication.name}:`, error.message));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, refreshIntervalMilliseconds).unref();
|
||||||
Reference in new issue
Block a user