Return All Data Lake Pipeline Runs from One Project

GET /api/atlas/v2/groups/{groupId}/pipelines/{pipelineName}/runs

Returns a list of past Data Lake Pipeline runs. To use this resource, the requesting API Key must have the Project Read Only role.

Path parameters

  • groupId string Required

    Unique 24-hexadecimal digit string that identifies your project. Use the /groups endpoint to retrieve all projects to which the authenticated user has access.

    NOTE: Groups and projects are synonymous terms. Your group id is the same as your project id. For existing groups, your group/project id remains the same. The resource and corresponding endpoints use the term groups.

    Minimum length is 24, maximum length is 24. Format should match the following pattern: ^([a-f0-9]{24})$.

  • pipelineName string Required

    Human-readable label that identifies the Data Lake Pipeline.

    Minimum length is 1, maximum length is 64. Format should match the following pattern: ^[^/\\ "$]{1,64}$.

Query parameters

  • envelope boolean

    Flag that indicates whether Application wraps the response in an envelope JSON object. Some API clients cannot access the HTTP response headers or status code. To remediate this, set envelope=true in the query. Endpoints that return a list of results use the results object as an envelope. Application adds the status parameter to the response body.

    Default value is false.

  • includeCount boolean

    Flag that indicates whether the response returns the total number of items (totalCount) in the response.

    Default value is true.

  • itemsPerPage integer

    Number of items that the response returns per page.

    Minimum value is 1, maximum value is 500. Default value is 100.

  • pageNum integer

    Number of the page that displays the current set of the total objects that the response returns.

    Minimum value is 1. Default value is 1.

  • pretty boolean

    Flag that indicates whether the response body should be in the prettyprint format.

    Default value is false.

  • createdBefore string(date-time)

    If specified, Atlas returns only Data Lake Pipeline runs initiated before this time and date.

Responses

  • 200 application/vnd.atlas.2023-01-01+json

    OK

    Hide response attributes Show response attributes object
    • results array[object]

      List of returned documents that MongoDB Cloud providers when completing this request.

      Run details of a Data Lake Pipeline.

      Hide results attributes Show results attributes object
      • _id string

        Unique 24-hexadecimal character string that identifies a Data Lake Pipeline run.

        Minimum length is 24, maximum length is 24. Format should match the following pattern: ^([a-f0-9]{24})$.

      • backupFrequencyType string

        Backup schedule interval of the Data Lake Pipeline.

        Values are HOURLY, DAILY, WEEKLY, MONTHLY, YEARLY, or ON_DEMAND.

      • createdDate string(date-time)

        Timestamp that indicates when the pipeline run was created.

      • datasetName string

        Human-readable label that identifies the dataset that Atlas generates during this pipeline run. You can use this dataset as a dataSource in a Federated Database collection.

      • datasetRetentionPolicy object

        Dataset Retention Policy for a Scheduled Data Lake Pipeline.

        Hide datasetRetentionPolicy attributes Show datasetRetentionPolicy attributes object
        • lastModifiedDate string(date-time)

          Date when retention policy was last modified.

        • units string Required

          Quantity of time in which the Data Lake Pipeline measures dataset retention.

          Values are DAYS, WEEKS, or MONTHS.

        • value integer(int32) Required

          Number that indicates the amount of days, weeks, or months that the Data Lake Pipeline will retain datasets.

          Minimum value is 1.

      • groupId string

        Unique 24-hexadecimal character string that identifies the project.

        Minimum length is 24, maximum length is 24. Format should match the following pattern: ^([a-f0-9]{24})$.

      • lastUpdatedDate string(date-time)

        Timestamp that indicates the last time that the pipeline run was updated.

      • phase string

        Processing phase of the Data Lake Pipeline.

        Values are SNAPSHOT, EXPORT, or INGESTION.

      • pipelineId string

        Unique 24-hexadecimal character string that identifies a Data Lake Pipeline.

        Minimum length is 24, maximum length is 24. Format should match the following pattern: ^([a-f0-9]{24})$.

      • scheduledDeletionDate string(date-time)

        Timestamp that indicates when the pipeline run will expire and its dataset will be deleted.

      • snapshotId string

        Unique 24-hexadecimal character string that identifies the snapshot of a cluster.

        Minimum length is 24, maximum length is 24. Format should match the following pattern: ^([a-f0-9]{24})$.

      • state string

        State of the pipeline run.

        Values are PENDING, IN_PROGRESS, DONE, FAILED, or DATASET_DELETED.

      • stats object

        Runtime statistics for this Data Lake Pipeline run.

        Hide stats attributes Show stats attributes object
        • bytesExported integer(int64)

          Total data size in bytes exported for this pipeline run.

        • numDocs integer(int64)

          Number of docs ingested for a this pipeline run.

    • totalCount integer(int32)

      Total number of documents available. MongoDB Cloud omits this value if includeCount is set to false.

      Minimum value is 0.

  • 400 application/json

    Bad Request.

    Hide response attributes Show response attributes object
    • detail string

      Describes the specific conditions or reasons that cause each type of error.

    • error integer(int32)

      HTTP status code returned with this error.

      External documentation
    • errorCode string

      Application error code returned with this error.

    • parameters array[object]

      Parameters used to give more information about the error.

    • reason string

      Application error message returned with this error.

  • 404 application/json

    Not Found.

    Hide response attributes Show response attributes object
    • detail string

      Describes the specific conditions or reasons that cause each type of error.

    • error integer(int32)

      HTTP status code returned with this error.

      External documentation
    • errorCode string

      Application error code returned with this error.

    • parameters array[object]

      Parameters used to give more information about the error.

    • reason string

      Application error message returned with this error.

  • 500 application/json

    Internal Server Error.

    Hide response attributes Show response attributes object
    • detail string

      Describes the specific conditions or reasons that cause each type of error.

    • error integer(int32)

      HTTP status code returned with this error.

      External documentation
    • errorCode string

      Application error code returned with this error.

    • parameters array[object]

      Parameters used to give more information about the error.

    • reason string

      Application error message returned with this error.

GET /api/atlas/v2/groups/{groupId}/pipelines/{pipelineName}/runs
atlas api dataLakePipelines listPipelineRuns --help
import (
	"os"
	"context"
	"log"
	sdk "go.mongodb.org/atlas-sdk/v20230101001/admin"
)

func main() {
	ctx := context.Background()
	clientID := os.Getenv("MONGODB_ATLAS_CLIENT_ID")
	clientSecret := os.Getenv("MONGODB_ATLAS_CLIENT_SECRET")

	// See https://dochub.mongodb.org/core/atlas-go-sdk-oauth
	client, err := sdk.NewClient(sdk.UseOAuthAuth(clientID, clientSecret))

	if err != nil {
		log.Fatalf("Error: %v", err)
	}

	params = &sdk.ListPipelineRunsApiParams{}
	sdkResp, httpResp, err := client.DataLakePipelinesApi.
		ListPipelineRunsWithParams(ctx, params).
		Execute()
}
curl --include --header "Authorization: Bearer ${ACCESS_TOKEN}" \
  --header "Accept: application/vnd.atlas.2023-01-01+json" \
  -X GET "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/pipelines/{pipelineName}/runs?pretty=true"
curl --user "${PUBLIC_KEY}:${PRIVATE_KEY}" \
  --digest --include \
  --header "Accept: application/vnd.atlas.2023-01-01+json" \
  -X GET "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/pipelines/{pipelineName}/runs?pretty=true"
Response examples (200)
{
  "links": [
    {
      "href": "https://cloud.mongodb.com/api/atlas",
      "rel": "self"
    }
  ],
  "results": [
    {
      "_id": "32b6e34b3d91647abb20e7b8",
      "backupFrequencyType": "HOURLY",
      "createdDate": "2025-05-04T09:42:00Z",
      "datasetName": "v1$atlas$snapshot$Cluster0$myDatabase$myCollection$19700101T000000Z",
      "datasetRetentionPolicy": {
        "lastModifiedDate": "2025-05-04T09:42:00Z",
        "units": "DAYS",
        "value": 42
      },
      "groupId": "32b6e34b3d91647abb20e7b8",
      "lastUpdatedDate": "2025-05-04T09:42:00Z",
      "phase": "SNAPSHOT",
      "pipelineId": "32b6e34b3d91647abb20e7b8",
      "scheduledDeletionDate": "2025-05-04T09:42:00Z",
      "snapshotId": "32b6e34b3d91647abb20e7b8",
      "state": "PENDING",
      "stats": {
        "bytesExported": 42,
        "numDocs": 42
      }
    }
  ],
  "totalCount": 42
}
Response examples (400)
{
  "error": 400,
  "detail": "(This is just an example, the exception may not be related to this endpoint) No provider AWS exists.",
  "reason": "Bad Request",
  "errorCode": "VALIDATION_ERROR"
}
Response examples (404)
{
  "error": 404,
  "detail": "(This is just an example, the exception may not be related to this endpoint) Cannot find resource AWS",
  "reason": "Not Found",
  "errorCode": "RESOURCE_NOT_FOUND"
}
Response examples (500)
{
  "error": 500,
  "detail": "(This is just an example, the exception may not be related to this endpoint)",
  "reason": "Internal Server Error",
  "errorCode": "UNEXPECTED_ERROR"
}