Skip to main content

SFTP JSON

This destination writes each stream to a newline-delimited JSON file in a directory on an SFTP server.

Prerequisites

  • An SFTP server reachable from your Airbyte deployment.
  • A username and password for that server. This destination authenticates with a password. Its configuration has no field for a private key, so it can't authenticate with an SSH key.
  • An existing directory on the server that the account can read from and write to. The connector doesn't create the directory for you.

Setup

Configure the following fields:

FieldRequiredDefaultDescription
HostYesHostname or IP address of the SFTP server.
PortNo22Port the SFTP server listens on.
UserYesUsername used to sign in to the server.
PasswordYesPassword for that username.
Destination pathYesAbsolute path to the directory where the connector writes files, such as /json_data.

When you test the connection, the connector writes a small file to Destination path and then deletes it, so the account needs write and delete permissions in that directory, not just read access.

Output files

The connector writes one file per stream and names it after the stream:

{destination_path}/airbyte_json_{stream_name}.jsonl

For example, if Destination path is /json_data and you sync a stream named users, the connector writes /json_data/airbyte_json_users.jsonl.

Each line in the file is a single JSON object holding the record exactly as the source emitted it. The connector doesn't add Airbyte metadata fields such as _airbyte_raw_id or _airbyte_extracted_at, so the file contains only your source data.

Because the file name comes from the stream name alone, two streams with the same name in different namespaces write to the same file. This destination doesn't support namespaces.

Supported sync modes

Sync modeSupported?
Full Refresh - OverwriteYes
Full Refresh - AppendYes
Full Refresh - Overwrite + DedupedNo
Incremental Sync - AppendYes
Incremental Sync - Append + DedupedNo

In overwrite mode, the connector deletes the stream's existing file at the start of the sync and writes a new one. In the append modes, it appends records to the end of the existing file and never rewrites earlier lines, so records from failed or partial syncs stay in the file. Deduplicate downstream if that matters to you.

Performance considerations

Throughput depends on your network connection to the SFTP server and how quickly the server accepts writes. The connector holds one open file handle per stream and writes records as it receives them, so it doesn't batch or compress data.

Troubleshooting

The connection test fails with an authentication error

Confirm that the server accepts password authentication for this account. This connector can't connect to servers that require public key authentication.

The connection test fails but the credentials are correct

Check that Destination path already exists and that the account can write to it and delete files in it. The connector doesn't create missing directories.

Passwords containing special characters

Version 0.2.16 and later percent-encode the username, password, host, and path before building the SFTP URI, so credentials containing characters such as #, @, /, or ? work as entered. On earlier versions, these characters truncated or corrupted the URI and syncs and connection tests failed. If you're on an earlier version, upgrade rather than changing the password.

Reference

Config fields reference

Field
Type
Property name
string
destination_path
string
host
string
password
string
username
integer
port

Changelog

Expand to review
VersionDatePull RequestSubject
0.2.162026-08-1378111Escape URI-reserved characters in credentials and paths, fixing syncs and connection checks for passwords containing characters such as #
0.2.152025-05-2760870Update dependencies
0.2.142025-05-1059809Update dependencies
0.2.132025-05-0359353Update dependencies
0.2.122025-04-2658727Update dependencies
0.2.112025-04-1958238Update dependencies
0.2.102025-04-1257592Update dependencies
0.2.92025-04-0557114Update dependencies
0.2.82025-03-2956615Update dependencies
0.2.72025-03-2256090Update dependencies
0.2.62025-03-0855369Update dependencies
0.2.52025-03-0154868Update dependencies
0.2.42025-02-2254265Update dependencies
0.2.32025-02-1553941Update dependencies
0.2.22025-02-0853405Update dependencies
0.2.12025-02-0152883Update dependencies
0.2.02024-10-1446873Migrated to Poetry and Airbyte Base Image
0.1.02022-11-244924🎉 New Destination: SFTP JSON