Quickstart

Ship your first bytes from edge to S3 in ~5 minutes

After creating an account, the EdgeMQ Quickstart wizard will guide you through:

Generate API Key - for use with an ingest endpoint
Connect S3 - by creating an IAM Role EdgeMQ can assume with write‑only access to that prefix.
Deploy Endpoint - a dedicated VM in an edge location of your choice
Send Test Data - verify the full edge ingest-to-S3 path.

This document provides a summary of each step.

1) Generate API Key

Generate an API key that will be used to authenticate against ingest endpoints:

2) Connect S3

Step 2 of the Quickstart will guide you through connecting to AWS S3 with a secure and least privilege posture, so that EdgeMQ can write data to your S3 bucket.

After completing the S3 form, EdgeMQ will generate trust and permissions policies that can be used to configure an IAM role.

Use STS `AssumeRole` with least‑privilege, write‑only access to your `bucket/prefix`. This keeps credentials short‑lived and fully controlled by your account. See Security posture below for rationale and alternatives.

The Console Quickstart will

Screenshot of Step 2 in the EdgeMQ console showing the S3 destination form. — Real-time deployment updates

2a) Trust policy (who may assume the role)

Set the principal to your EdgeMQ Control‑Plane AWS principal and require the ExternalId shown in the Console when creating the Destination.

Security Note — Why ExternalId matters

The ExternalId prevents the confused deputy problem—a cross‑account security issue where an attacker could trick EdgeMQ into writing to their bucket using your role. By requiring a unique ExternalId that only you and EdgeMQ know, your role cannot be assumed by anyone else, even if they know your AWS account ID and role name.

{
  "Version":: "2012-10-17",
  "Statement":: [
    {
      "Effect":: "Allow",
      "Principal":: { "AWS":: "arn:aws:iam::<CP_ACCOUNT_ID>:user/edgemq-ingest" },
      "Action":: "sts:AssumeRole",
      "Condition":: {
        "StringEquals":: { "sts:ExternalId":: "<EXTERNAL_ID_FROM_CONSOLE>" }
      }
    }
  ]
}

2b) Permissions policy (write‑only to your prefix)

Allow only the actions required for multipart uploads and a small validation object the Console writes.

Replace placeholders with your bucket/prefix.

{
  "Version":: "2012-10-17",
  "Statement":: [
    {
      "Sid":: "BucketMeta",
      "Effect":: "Allow",
      "Action":: ["s3:GetBucketLocation"],
      "Resource":: "arn:aws:s3:::<your-bucket-name>"
    },
    {
      "Sid":: "ObjectWrite",
      "Effect":: "Allow",
      "Action":: [
        "s3:PutObject",
        "s3:AbortMultipartUpload",
        "s3:CreateMultipartUpload",
        "s3:UploadPart",
        "s3:DeleteObject"
      ],
      "Resource":: [
        "arn:aws:s3:::<your-bucket-name>/<prefix>/*",
        "arn:aws:s3:::<your-bucket-name>/<prefix>._edgemq_validation/*"
      ]
    }
  ]
}

Using KMS? Also allow `kms:Encrypt` and `kms:GenerateDataKey` on your CMK and enter the KMS Key ARN in the Console.

3) Deploy Endpoint

Step 3 of the Quickstart will deploy one or more ingest endpoints. An ingest endpoint is a dedicated VM, NVMe-backed WAL volume (no shared disk) with isolated process boundaries. Once deployed, an ingest endpoint will provide an ingest URL to be used for sending data.

4) Test and Verify

Step 4 provides a test utility that will send a data payload to your deployed endpoint.

Test with curl:

head -c 1024 /dev/urandom | \
curl -sS -o /dev/null -w 'HTTP %{http_code}\n' \
  -X POST https://<your_endpoint>/v1/ingest \
  -H 'Content-Type: application/octet-stream' \
  -H 'X-API-Key: <your-api-key>' \
  --data-binary @-

Look in S3 under your prefix. Objects look like:

<region>/<instance-id>/<epoch>/segments/seg-000000NN.wal.zst

Security posture

Principles

Short‑lived credentials via STS `AssumeRole`; no long‑lived keys stored by EdgeMQ.
Least‑privilege: write‑only to your exact `bucket/prefix` (no read/list/get).
Isolation by environment (dev/stage/prod) via distinct roles and prefixes.

Why Role over User?

Roles issue temporary credentials and are easy to revoke or rotate centrally.
Users create long‑lived keys that you must store and rotate—higher risk.
If you must use a User, apply the same least‑privilege policy and rotate keys.

Permissions scope

Trust policy requires the EdgeMQ control‑plane principal and `ExternalId` (prevents confused deputy attacks—see trust policy section above).
Permissions policy allows multipart writes only to `arn:aws:s3:::<bucket>/<prefix>/*` and the validation path `arn:aws:s3:::<bucket>/<prefix>._edgemq_validation/*`.
Using KMS? Allow only `kms:Encrypt` and `kms:GenerateDataKey` on your CMK.

Revocation & audit

To revoke, remove our principal from the trust policy or detach the policy.
Keep bucket encryption on (SSE‑S3 or KMS) and enable CloudTrail/S3 server access logs.