# Storage
> Write files to a named disk, stream large payloads and keep paths relative to storage. Add Media when files need durable identities and ownership metadata.
- Package: `@db3.ai/app/storage`
- Canonical page: [https://db3.ai/docs/storage](https://db3.ai/docs/storage)
- Markdown: [https://db3.ai/docs/storage.md](https://db3.ai/docs/storage.md)
- Framework source of truth: `packages/app/src/storage/README.md`
## Configure a disk
Use `storage.default` and `storage.disks` in App options. The example creates a local `exports` disk in a temporary directory. In an application, choose a persistent absolute root or deliberately resolve one relative to the process working directory.
Without configuration, the local disk writes under `storage/app` relative to the current working directory. That is convenient locally, but a container filesystem is not automatically durable storage. A standalone script can also construct `Storage` directly.
- [Install the runtime and development tools](https://db3.ai/docs/installation.md)
## Copy the export example
Use the independent app directory from Installation. This lab does not need a database, an S3 account or an HTTP server.
### Copy the shipped example
```bash
mkdir -p examples
cp -R node_modules/@db3.ai/app/src/storage/examples/. examples/
```
## Run it
The script writes a JSON report and streamed CSV, reads them, lists their relative paths and deletes the report. The result should include `traversalRejected: true` and `deleted: true`. Its temporary disk is removed afterwards.
### Run the lab
```bash
npx tsx examples/runStorage.ts
```
## Write and read files
`write()` and `put()` write text, bytes or supported stream contents. Use `readToString()` for text and `readToBuffer()` or `get()` when you deliberately want the whole file in memory. `read()` returns a stream, not a string.
Choose a named disk with `app().storage.disk("exports")`; normal calls on `app().storage` use the default. `drive()` is an alias for `disk()`.
### examples/runStorage.ts
```typescript
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { Readable } from 'node:stream';
import { pathToFileURL } from 'node:url';
import { App } from '@db3.ai/app/server';
/**
* Writes, reads, lists and removes an export on an isolated real local disk.
*
* This runnable lab owns its temporary directory, not application storage.
* The same storage calls can use a configured app().storage in feature code.
*
* @returns Stable observations; no temporary absolute path is exposed.
*/
export async function runStorage() {
const root = await mkdtemp(join(tmpdir(), 'db3-storage-guide-'));
const application = new App({ storage: { default: 'exports', disks: { exports: { driver: 'local', root } } } });
try {
const disk = application.storage.disk('exports');
await disk.write('reports/latest.json', JSON.stringify({ status: 'ready' }), { mimeType: 'application/json' });
const report = JSON.parse(await disk.readToString('reports/latest.json'));
await disk.writeStream('reports/export.csv', Readable.from(['name\n', 'Ada\n']), { mimeType: 'text/csv' });
const csv = await disk.readToString('reports/export.csv');
const files = (await disk.list('reports').toArray()).filter(entry => entry.isFile).map(entry => entry.path);
let traversalRejected = false;
try { await disk.write('../outside.txt', 'not allowed'); } catch { traversalRejected = true; }
await disk.delete('reports/latest.json');
return { report, csv, files, traversalRejected, deleted: await disk.missing('reports/latest.json') };
} finally {
try { await application.close(); } finally { await rm(root, { recursive: true, force: true }); }
}
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
console.log(JSON.stringify(await runStorage(), null, 2));
}
```
## Stream uploads and exports
Use `writeStream()` and `readStream()` for large payloads. Consume the returned stream or pass it to your HTTP response, with proper stream-error handling. The wrapper does not buffer the whole file before passing it to the driver.
The example uses a short CSV stream so the result is easy to inspect. Apply upload limits, client-disconnect handling and temporary-upload cleanup in the HTTP adapter. A streamed API alone does not provide those policies.
## List and inspect stored files
`exists()` and `missing()` check a path. `size()`, `lastModified()` and `mimeType()` expose file metadata. `list()` returns a lazy async iterable; `toArray()` intentionally collects it. Use `deep: true` only when descendants are needed.
`delete()` removes the specified file. Treat listing paths as disk-relative names, not application IDs or proof that the caller may read them. Use a scoped Media record when you need that ownership layer.
## Use S3-compatible storage
Configure `driver: "s3"`, `bucket` and `region`. Set `endpoint` for an S3-compatible service, and `forcePathStyle` when that provider requires it. Keep credentials in your environment or the supported provider credential chain, never browser configuration.
The service-owned reference includes the full S3 disk configuration. The local lab tests the framework workflow, not credentials, permissions, listing consistency or multipart behaviour on your remote provider. Run a small upload/read/delete smoke test in a dedicated bucket before production.
## Paths, URLs and private delivery
File paths are relative to the disk root. Absolute paths, parent traversal and null bytes are rejected. Empty paths are allowed for listing the root, not for writing a file. This is path validation, not a sandbox against untrusted filesystem changes or an authorization system.
`path()` returns a local filesystem path; remote disks cannot supply one. `url()` needs a configured public URL or provider support and only constructs a URL. It does not create a route, sign a private download or grant permission.
For private downloads, authorize the caller first, then stream the allowed file with deliberate content type and disposition headers. Never publish a private storage root just to make `url()` work.
- [Add durable file IDs and library scopes](https://db3.ai/docs/media.md)
## Testing
Create a `tests/storage` directory and save the test below as `runStorage.test.ts`. It imports the example you copied into `examples/`. Keep the same folder layout so the relative import resolves.
Run from the application root with the development dependencies from Installation. These are consumer tests, not commands that assume a framework checkout.
### tests/storage/runStorage.test.ts
```typescript
import { expect, it } from 'vitest';
import { runStorage } from '../../examples/runStorage';
it('runs the storage guide on a real isolated local disk', async () => {
expect(await runStorage()).toEqual({
report: { status: 'ready' }, csv: 'name\nAda\n',
files: ['reports/export.csv', 'reports/latest.json'],
traversalRejected: true, deleted: true,
});
});
```
## Run and extend the test
The copied test uses a real temporary local disk and removes it in `finally`. Add a named disk or another file and assert the result. Use dedicated test storage, not a production bucket, for destructive cases.
### Run your copied test and check types
```bash
npx vitest run tests/storage/runStorage.test.ts
npx tsc --noEmit --target ES2022 --module ESNext --moduleResolution Bundler --types node --skipLibCheck examples/*.ts
```
## Coverage and reference
Tested here: named local disks, JSON/text, streamed writes, listing, traversal rejection and deletion. The linked export recipe tests producer interruption and recovery; the private-file guide tests authenticated downloads.
Still to build: socket-disconnect and provider-specific S3 integration trials. Those are not implied by local labs passing.
- [Full Storage API](https://db3.ai/docs/storage-api.md)
- [Stream failure and recovery](https://db3.ai/docs/cookbook-streams.md)
- [Authenticated files](https://db3.ai/docs/guide-files.md)
## Behavioural verification
Writes JSON and a streamed CSV, lists the files, rejects traversal and removes a file on a real local disk.
- Behaviour test: `packages/app/src/storage/tests/examples/runStorage.test.ts`
- Repository test command (framework checkout only): `npm run test:service --workspace @platform/app -- storage --maxWorkers=1`
- In an installed application, use the walkthrough commands instead of this repository test.
- Expected outcome: The guide test passes against the real framework components.
- Environment: Node.js and an isolated temporary directory; no database or remote storage account.
## Related documentation
- [Install the framework](https://db3.ai/docs/installation.md): Install one runtime package, then build a small HTTP app. Add a database and other services when your feature needs them.
- [Media](https://db3.ai/docs/media.md): Give files durable IDs and project-scoped libraries. Keep storage paths, browser folders and permission checks separate.
- [Create your first app](https://db3.ai/docs/create-app.md): Start an HTTP server, return a JSON response, reject invalid input and test it without opening a port.
## Framework-owned source: `packages/app/src/storage/README.md`
This is the exact source document captured by the documentation build. Use it for detailed API and workflow guidance, subject to the public package exports and behavioural evidence identified above.
````markdown
# @db3.ai/app/storage
## Stream failure and recovery lab
Copy `src/storage/examples/` from the installed package into `examples/` and run
`npx tsx examples/runStreamExport.ts`. The lab streams 50,000 rows, interrupts a
staged write, preserves the last successful export and removes partial bytes.
It needs only temporary local storage and removes that directory afterwards.
`tests/examples/runStreamExport.test.ts` is the consumer-copyable behavioral
check. Remote moves and HTTP disconnects need separate adapter/driver tests.
`@db3.ai/app/storage` provides a Laravel-style storage service with named
disks. App code can write to the default disk through `app.storage` or select a
configured disk with `app.storage.disk(name)`.
The storage service is the lower-level file API. Managed file records, ULID
references, and database-backed file metadata should sit above this layer.
## Configure disks
```ts
import { App } from '@db3.ai/app';
const app = new App({
storage: {
default: 'local',
disks: {
local: {
driver: 'local',
root: 'storage/app',
},
agent: {
driver: 'local',
root: 'storage/agent',
url: 'https://example.test/storage/agent',
},
spaces: {
driver: 's3',
bucket: 'example-assets',
region: 'nyc3',
endpoint: 'https://nyc3.digitaloceanspaces.com',
url: 'https://example-assets.nyc3.digitaloceanspaces.com',
accessKeyId: process.env.SPACES_ACCESS_KEY_ID,
secretAccessKey: process.env.SPACES_SECRET_ACCESS_KEY,
},
},
},
});
```
If no storage config is supplied, `local` is available and writes under
`storage/app` relative to the process working directory. The local driver is
backed by Flystorage's `FileStorage` and `LocalStorageAdapter`.
## Day-To-Day Usage
Most app code should use the default disk unless it has a good reason to choose
a specific backend. `write(...)` and `put(...)` do the same thing. `put(...)`
matches Laravel naming, while `write(...)` is convenient in the REPL.
### Write A Text File
```ts
await app().storage.write('my/test-file.txt', 'hello i am a test file');
```
Or, using the Laravel-style name:
```ts
await app().storage.put('my/test-file.txt', 'hello i am a test file');
```
### Read A Text File
```ts
const text = await app().storage.readToString('my/test-file.txt');
```
`getText(...)` is also available when you want to pass a Node text encoding.
### Read A Stream
`read(...)` follows Flystorage's API and returns a readable stream:
```ts
const stream = await app().storage.read('my/test-file.txt');
```
Use `get(...)` when you intentionally want the whole file buffered:
```ts
const buffer = await app().storage.readToBuffer('my/test-file.txt');
const bytes = await app().storage.readToUint8Array('my/test-file.txt');
```
`get(...)` is also available as a shorter alias for `readToBuffer(...)`.
### Write JSON
```ts
await app().storage.write('reports/latest.json', JSON.stringify({
status: 'ready',
}, null, '\t'), {
mimeType: 'application/json',
});
```
### Write Bytes
```ts
await app().storage.write('images/avatar.png', imageBytes, {
mimeType: 'image/png',
visibility: 'private',
});
```
### Check, Delete, And Inspect Files
```ts
await app().storage.exists('my/test-file.txt');
await app().storage.missing('my/missing-file.txt');
await app().storage.size('my/test-file.txt');
await app().storage.lastModified('my/test-file.txt');
await app().storage.mimeType('images/avatar.png');
await app().storage.delete('my/test-file.txt');
```
### List Directory Contents
`list(...)` returns a lazy, provider-neutral async listing for local and remote
disks. Set `deep` when descendants below the immediate directory are needed:
```ts
const listing = app().storage.disk('local').list('reports', {
deep: true,
});
for await (const entry of listing) {
console.log(entry.type, entry.path, entry.lastModified);
}
```
Collect a listing when the complete result is intentionally needed in memory:
```ts
const entries = await app().storage.disk('local').list('reports', {
deep: true,
}).toArray();
```
Pass an empty path or omit it to list from the disk root. Listings expose the
same framework entry shape for every driver; callers do not need to know whether
the selected disk is local or S3-compatible.
For local disks, `path(...)` returns the absolute filesystem path:
```ts
app().storage.path('my/test-file.txt');
```
Remote disks such as S3 do not expose local paths, so use `url(...)` when a
public URL is configured:
```ts
await app().storage.url('images/avatar.png');
```
### Use A Named Disk
```ts
await app().storage.disk('agent').write('generated-images/example.txt', 'hello agent disk');
const text = await app().storage.disk('agent').getText('generated-images/example.txt');
```
`drive(name)` is also available as an alias for `disk(name)`:
```ts
await app().storage.drive('agent').write('notes/example.txt', 'stored on the agent disk');
```
## Try It In Tinker
From the repo root, start the CLI REPL:
```sh
npm run tinker
```
An application tinker context can expose both `app` and `app()` styles. Top-level
`await` is supported, so each command waits for the storage operation to finish
before the prompt returns:
```ts
await app().storage.write('my/test-file.txt', 'hello i am a test file')
await app().storage.read('my/test-file.txt')
await app().storage.readToString('my/test-file.txt')
await app().storage.readToBuffer('my/test-file.txt')
await app().storage.readToUint8Array('my/test-file.txt')
await app().storage.exists('my/test-file.txt')
await app().storage.delete('my/test-file.txt')
```
If you prefer property access, this works too:
```ts
await app.storage.write('my/test-file.txt', 'hello from app.storage')
```
## Use The Default Disk
```ts
await app.storage.put('reports/latest.json', JSON.stringify({
status: 'ready',
}));
const text = await app.storage.getText('reports/latest.json');
```
## Use A Named Disk
```ts
const agent = app.storage.disk('agent');
await agent.write('generated-images/example.png', imageBytes, {
mimeType: 'image/png',
visibility: 'private',
});
const bytes = await agent.get('generated-images/example.png');
```
## Stream Large Files
Use explicit stream methods when files should not be buffered into memory:
```ts
await app.storage.disk('agent').writeStream('uploads/source-video.mp4', requestStream, {
mimeType: 'video/mp4',
});
const stream = await app.storage.disk('agent').readStream('uploads/source-video.mp4');
```
`read(...)` is also available and matches Flystorage's stream-returning method:
```ts
const stream = await app.storage.disk('agent').read('uploads/source-video.mp4');
```
These methods are backed by Flystorage streams, so they are the preferred API
for large uploads, exports, and future remote disks such as S3. The framework
wrapper passes stream objects through to the driver and does not buffer them
before writing or reading.
## S3-Compatible Disks
Use `driver: 's3'` for AWS S3 and services that support the S3 protocol:
```ts
const storage = new Storage({
default: 'spaces',
disks: {
spaces: {
driver: 's3',
bucket: 'example-assets',
region: 'nyc3',
endpoint: 'https://nyc3.digitaloceanspaces.com',
url: 'https://example-assets.nyc3.digitaloceanspaces.com',
accessKeyId: env.required('SPACES_ACCESS_KEY_ID'),
secretAccessKey: env.required('SPACES_SECRET_ACCESS_KEY'),
},
},
});
```
`endpoint` is optional for AWS S3 and useful for providers such as DigitalOcean
Spaces, MinIO, and other S3-compatible services. Use `forcePathStyle: true`
when the provider expects path-style bucket URLs.
## Path safety
Storage paths are always relative to the disk root. Absolute paths, null bytes,
and parent-directory traversal are rejected before the driver touches the
filesystem. File operations reject empty paths; `list('')` deliberately allows
an empty path to list the disk root.
## Runnable example and testing
The [storage lab](./examples/runStorage.ts) writes and reads JSON and a streamed
CSV, lists the resulting paths, checks traversal rejection and deletes a file.
It uses a real temporary local disk, removes that directory afterwards and does
not need SQL or cloud credentials. Copy the shipped example into `examples`
and run `npx tsx examples/runStorage.ts` in the consuming app.
The Storage website page includes the exact Vitest test for that copied file.
Framework contributors run `npm run test:service --workspace packages/app -- storage`.
Local behaviour is not evidence that an S3 account, bucket policy or interrupted
upload works correctly. Verify those against dedicated provider test resources.
## Local disk URLs
The local driver can build URLs only when `url` is configured:
```ts
const url = await app.storage.disk('agent').url('generated-images/example.png');
```
This method only returns the URL. Routes remain responsible for authentication,
ownership checks, headers, and response shaping.
````
## Guidance for AI tools
Use the documented public import `@db3.ai/app/storage` and its exported types. Prefer the source-backed examples and behavioural outcomes above over invented APIs or source-relative internal imports.