From bf0dcbea82f1ce043734fbfefa80cdbd3c38ddff Mon Sep 17 00:00:00 2001 From: Robert Lin Date: Fri, 10 Apr 2026 16:19:22 -0700 Subject: [PATCH 1/3] object-storage: add new 'sourcegraph' bucket --- .../external-services/object-storage.mdx | 175 ++++++++++++------ 1 file changed, 115 insertions(+), 60 deletions(-) diff --git a/docs/self-hosted/external-services/object-storage.mdx b/docs/self-hosted/external-services/object-storage.mdx index d78b4591a..3b00b443e 100644 --- a/docs/self-hosted/external-services/object-storage.mdx +++ b/docs/self-hosted/external-services/object-storage.mdx @@ -4,115 +4,170 @@ By default, Sourcegraph will use a `sourcegraph/blobstore` server bundled with t You can alternatively configure your instance to instead store this data in an S3 or GCS bucket. Doing so may decrease your hosting costs as persistent volumes are often more expensive than the same storage space in an object store service. +## `sourcegraph` bucket + + + Starting in Sourcegraph 7.2, self-hosted Sourcegraph instances using S3 or + GCS object storage should now provision an additional bucket named + `sourcegraph`. Sourcegraph currently reports a warning when this bucket is + not present, and it will become required for new features in a future + release. No action is required if you are using the default + `sourcegraph/blobstore`. + + +The `sourcegraph` bucket is intended to be the single bucket for new Sourcegraph features. Instead of creating one bucket per feature, new features store objects under namespaced key prefixes within this bucket. + +Existing buckets for code graph indexes and search jobs remain in use. This change ensures future features can be enabled without requiring a new bucket for each feature. + +### Using S3 for the `sourcegraph` bucket + +Set the following environment variables to target an S3 bucket for shared Sourcegraph uploads. + +- `SOURCEGRAPH_UPLOAD_BACKEND=S3` +- `SOURCEGRAPH_UPLOAD_BUCKET=sourcegraph` (default) +- `SOURCEGRAPH_UPLOAD_AWS_ENDPOINT=https://s3.us-east-1.amazonaws.com` +- `SOURCEGRAPH_UPLOAD_AWS_ACCESS_KEY_ID=` +- `SOURCEGRAPH_UPLOAD_AWS_SECRET_ACCESS_KEY=` +- `SOURCEGRAPH_UPLOAD_AWS_SESSION_TOKEN=` (optional) +- `SOURCEGRAPH_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) +- `SOURCEGRAPH_UPLOAD_AWS_USE_PATH_STYLE=false` (optional) +- `SOURCEGRAPH_UPLOAD_AWS_REGION=us-east-1` (default) + +### Using GCS for the `sourcegraph` bucket + +Set the following environment variables to target a GCS bucket for shared Sourcegraph uploads. + +- `SOURCEGRAPH_UPLOAD_BACKEND=GCS` +- `SOURCEGRAPH_UPLOAD_BUCKET=sourcegraph` (default) +- `SOURCEGRAPH_UPLOAD_GCP_PROJECT_ID=` +- `SOURCEGRAPH_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` +- `SOURCEGRAPH_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` + +### Automatically provision the `sourcegraph` bucket + +If you would like to allow your Sourcegraph instance to manage the target bucket configuration, set the following environment variable: + + + This requires additional bucket-management permissions from your configured + storage vendor (AWS or GCP). + + +- `SOURCEGRAPH_UPLOAD_MANAGE_BUCKET=true` + ## Code Graph Indexes To target a managed object storage service for storing [code graph index uploads](../../code-navigation/precise-code-navigation), you will need to set a handful of environment variables for configuration and authentication to the target service. -- If you are running a `sourcegraph/server` deployment, set the environment variables on the server container -- If you are running via Docker-compose or Kubernetes, set the environment variables on the `frontend`, `worker`, and `precise-code-intel-worker` containers +- If you are running a `sourcegraph/server` deployment, set the environment variables on the server container +- If you are running via Docker-compose or Kubernetes, set the environment variables on the `frontend`, `worker`, and `precise-code-intel-worker` containers -### Using S3 +### Using S3 for the Code Graph Indexes bucket To target an S3 bucket you've already provisioned, set the following environment variables. Authentication can be done through [an access and secret key pair](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) (and optional session token), or via the EC2 metadata API. - {' '} - Never commit AWS access keys in Git. You should consider using a secret handling - service offered by your cloud provider.{' '} + Never commit AWS access keys in Git. You should consider using a secret + handling service offered by your cloud provider. -- `PRECISE_CODE_INTEL_UPLOAD_BACKEND=S3` -- `PRECISE_CODE_INTEL_UPLOAD_BUCKET=` -- `PRECISE_CODE_INTEL_UPLOAD_AWS_ENDPOINT=https://s3.us-east-1.amazonaws.com` -- `PRECISE_CODE_INTEL_UPLOAD_AWS_ACCESS_KEY_ID=` -- `PRECISE_CODE_INTEL_UPLOAD_AWS_SECRET_ACCESS_KEY=` -- `PRECISE_CODE_INTEL_UPLOAD_AWS_SESSION_TOKEN=` (optional) -- `PRECISE_CODE_INTEL_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) -- `PRECISE_CODE_INTEL_UPLOAD_AWS_REGION=us-east-1` (default) +- `PRECISE_CODE_INTEL_UPLOAD_BACKEND=S3` +- `PRECISE_CODE_INTEL_UPLOAD_BUCKET=` +- `PRECISE_CODE_INTEL_UPLOAD_AWS_ENDPOINT=https://s3.us-east-1.amazonaws.com` +- `PRECISE_CODE_INTEL_UPLOAD_AWS_ACCESS_KEY_ID=` +- `PRECISE_CODE_INTEL_UPLOAD_AWS_SECRET_ACCESS_KEY=` +- `PRECISE_CODE_INTEL_UPLOAD_AWS_SESSION_TOKEN=` (optional) +- `PRECISE_CODE_INTEL_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) +- `PRECISE_CODE_INTEL_UPLOAD_AWS_REGION=us-east-1` (default) - {' '} - If a non-default region is supplied, ensure that the subdomain of the endpoint - URL (_the `AWS_ENDPOINT` value_) matches the target region.{' '} + If a non-default region is supplied, ensure that the subdomain of the + endpoint URL (_the `AWS_ENDPOINT` value_) matches the target region. - {' '} - You don't need to set the `PRECISE_CODE_INTEL_UPLOAD_AWS_ACCESS_KEY_ID` environment - variable when using `PRECISE_CODE_INTEL_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` - because role credentials will be automatically resolved. Attach the IAM role - to the EC2 instances hosting the `frontend`, `worker`, and `precise-code-intel-worker` - containers in a multi-node environment.{' '} + You don't need to set the `PRECISE_CODE_INTEL_UPLOAD_AWS_ACCESS_KEY_ID` + environment variable when using + `PRECISE_CODE_INTEL_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` because role + credentials will be automatically resolved. Attach the IAM role to the EC2 + instances hosting the `frontend`, `worker`, and `precise-code-intel-worker` + containers in a multi-node environment. -### Using GCS +### Using GCS for the Code Graph Indexes bucket To target a GCS bucket you've already provisioned, set the following environment variables. Authentication is done through a service account key, supplied as either a path to a volume-mounted file, or the contents read in as an environment variable payload. -- `PRECISE_CODE_INTEL_UPLOAD_BACKEND=GCS` -- `PRECISE_CODE_INTEL_UPLOAD_BUCKET=` -- `PRECISE_CODE_INTEL_UPLOAD_GCP_PROJECT_ID=` -- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` -- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` +- `PRECISE_CODE_INTEL_UPLOAD_BACKEND=GCS` +- `PRECISE_CODE_INTEL_UPLOAD_BUCKET=` +- `PRECISE_CODE_INTEL_UPLOAD_GCP_PROJECT_ID=` +- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` +- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` -### Provisioning buckets +### Automatically provision the Code Graph Indexes bucket If you would like to allow your Sourcegraph instance to control the creation and lifecycle configuration management of the target buckets, set the following environment variables: -- `PRECISE_CODE_INTEL_UPLOAD_MANAGE_BUCKET=true` -- `PRECISE_CODE_INTEL_UPLOAD_TTL=168h` (default) + + This requires additional bucket-management permissions from your configured + storage vendor (AWS or GCP). + + +- `PRECISE_CODE_INTEL_UPLOAD_MANAGE_BUCKET=true` +- `PRECISE_CODE_INTEL_UPLOAD_TTL=168h` (default) ## Search Job Results To target a third party managed object storage service for storing [search job results](../../code-search/types/search-jobs), you must set a handful of environment variables for configuration and authentication to the target service. -- If you are running a `sourcegraph/server` deployment, set the environment variables on the server container -- If you are running via Docker-compose or Kubernetes, set the environment variables on the `frontend` and `worker` containers +- If you are running a `sourcegraph/server` deployment, set the environment variables on the server container +- If you are running via Docker-compose or Kubernetes, set the environment variables on the `frontend` and `worker` containers -### Using S3 +### Using S3 for the Search Job Results bucket Set the following environment variables to target an S3 bucket you've already provisioned. Authentication can be done through [an access and secret key pair](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) (and optionally through session token) or via the EC2 metadata API. - {' '} - Never commit AWS access keys in Git. You should consider using a secret handling - service offered by your cloud provider. + Never commit AWS access keys in Git. You should consider using a secret + handling service offered by your cloud provider. -- `SEARCH_JOBS_UPLOAD_BACKEND=S3` -- `SEARCH_JOBS_UPLOAD_BUCKET=` -- `SEARCH_JOBS_UPLOAD_AWS_ENDPOINT=https://s3.us-east-1.amazonaws.com` -- `SEARCH_JOBS_UPLOAD_AWS_ACCESS_KEY_ID=` -- `SEARCH_JOBS_UPLOAD_AWS_SECRET_ACCESS_KEY=` -- `SEARCH_JOBS_UPLOAD_AWS_SESSION_TOKEN=` (optional) -- `SEARCH_JOBS_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) -- `SEARCH_JOBS_UPLOAD_AWS_REGION=us-east-1` (default) +- `SEARCH_JOBS_UPLOAD_BACKEND=S3` +- `SEARCH_JOBS_UPLOAD_BUCKET=` +- `SEARCH_JOBS_UPLOAD_AWS_ENDPOINT=https://s3.us-east-1.amazonaws.com` +- `SEARCH_JOBS_UPLOAD_AWS_ACCESS_KEY_ID=` +- `SEARCH_JOBS_UPLOAD_AWS_SECRET_ACCESS_KEY=` +- `SEARCH_JOBS_UPLOAD_AWS_SESSION_TOKEN=` (optional) +- `SEARCH_JOBS_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) +- `SEARCH_JOBS_UPLOAD_AWS_REGION=us-east-1` (default) - {' '} - If a non-default region is supplied, ensure that the subdomain of the endpoint - URL (the `AWS_ENDPOINT` value) matches the target region. + If a non-default region is supplied, ensure that the subdomain of the + endpoint URL (the `AWS_ENDPOINT` value) matches the target region. - {' '} You don't need to set the `SEARCH_JOBS_UPLOAD_AWS_ACCESS_KEY_ID` environment - variable when using `SEARCH_JOBS_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` because - role credentials will be automatically resolved. + variable when using `SEARCH_JOBS_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` + because role credentials will be automatically resolved. -### Using GCS +### Using GCS for the Search Job Results bucket Set the following environment variables to target a GCS bucket you've already provisioned. Authentication is done through a service account key, either as a path to a volume-mounted file or the contents read in as an environment variable payload. -- `SEARCH_JOBS_UPLOAD_BACKEND=GCS` -- `SEARCH_JOBS_UPLOAD_BUCKET=` -- `SEARCH_JOBS_UPLOAD_GCP_PROJECT_ID=` -- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` -- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` +- `SEARCH_JOBS_UPLOAD_BACKEND=GCS` +- `SEARCH_JOBS_UPLOAD_BUCKET=` +- `SEARCH_JOBS_UPLOAD_GCP_PROJECT_ID=` +- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` +- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` -### Provisioning buckets +### Automatically provision the Search Job Results bucket If you would like to allow your Sourcegraph instance to control the creation and lifecycle configuration management of the target buckets, set the following environment variables: -- `SEARCH_JOBS_UPLOAD_MANAGE_BUCKET=true` + + This requires additional bucket-management permissions from your configured + storage vendor (AWS or GCP). + + +- `SEARCH_JOBS_UPLOAD_MANAGE_BUCKET=true` From 419e33656d550a3dfac78c661f705c5d94642772 Mon Sep 17 00:00:00 2001 From: Robert Lin Date: Fri, 10 Apr 2026 16:32:04 -0700 Subject: [PATCH 2/3] single-bucket mode --- .../external-services/object-storage.mdx | 32 ++++++++++++++++--- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/docs/self-hosted/external-services/object-storage.mdx b/docs/self-hosted/external-services/object-storage.mdx index 3b00b443e..0f8686f2c 100644 --- a/docs/self-hosted/external-services/object-storage.mdx +++ b/docs/self-hosted/external-services/object-storage.mdx @@ -4,14 +4,24 @@ By default, Sourcegraph will use a `sourcegraph/blobstore` server bundled with t You can alternatively configure your instance to instead store this data in an S3 or GCS bucket. Doing so may decrease your hosting costs as persistent volumes are often more expensive than the same storage space in an object store service. + + Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), new instances can configure only the + [`sourcegraph` bucket](#sourcegraph-bucket), and Sourcegraph will use that + single bucket for all features. If a separate bucket is needed for Code Graph + Indexes or Search Job Results, that can still be configured, but we recommend + using one bucket. + + + + ## `sourcegraph` bucket - Starting in Sourcegraph 7.2, self-hosted Sourcegraph instances using S3 or + Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), self-hosted Sourcegraph instances using S3 or GCS object storage should now provision an additional bucket named - `sourcegraph`. Sourcegraph currently reports a warning when this bucket is - not present, and it will become required for new features in a future - release. No action is required if you are using the default + [`sourcegraph`](#sourcegraph-bucket). Sourcegraph currently reports a warning + when this bucket is not present, and it will become required for new features + in a future release. No action is required if you are using the default `sourcegraph/blobstore`. @@ -58,6 +68,13 @@ If you would like to allow your Sourcegraph instance to manage the target bucket To target a managed object storage service for storing [code graph index uploads](../../code-navigation/precise-code-navigation), you will need to set a handful of environment variables for configuration and authentication to the target service. + + Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), new instances can configure only the + [`sourcegraph` bucket](#sourcegraph-bucket), and Sourcegraph will use that + single bucket for all features. If a separate bucket is needed for Code Graph + Indexes, that can still be configured, but we recommend using one bucket. + + - If you are running a `sourcegraph/server` deployment, set the environment variables on the server container - If you are running via Docker-compose or Kubernetes, set the environment variables on the `frontend`, `worker`, and `precise-code-intel-worker` containers @@ -119,6 +136,13 @@ If you would like to allow your Sourcegraph instance to control the creation and To target a third party managed object storage service for storing [search job results](../../code-search/types/search-jobs), you must set a handful of environment variables for configuration and authentication to the target service. + + Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), new instances can configure only the + [`sourcegraph` bucket](#sourcegraph-bucket), and Sourcegraph will use that + single bucket for all features. If a separate bucket is needed for Search Job + Results, that can still be configured, but we recommend using one bucket. + + - If you are running a `sourcegraph/server` deployment, set the environment variables on the server container - If you are running via Docker-compose or Kubernetes, set the environment variables on the `frontend` and `worker` containers From 3430bed4b85c4b1bd4a343fd750497fcba857227 Mon Sep 17 00:00:00 2001 From: Robert Lin Date: Mon, 13 Apr 2026 18:54:18 -0700 Subject: [PATCH 3/3] updates --- .../external-services/object-storage.mdx | 68 +++++++++++-------- 1 file changed, 39 insertions(+), 29 deletions(-) diff --git a/docs/self-hosted/external-services/object-storage.mdx b/docs/self-hosted/external-services/object-storage.mdx index 0f8686f2c..bb14fc928 100644 --- a/docs/self-hosted/external-services/object-storage.mdx +++ b/docs/self-hosted/external-services/object-storage.mdx @@ -6,56 +6,58 @@ You can alternatively configure your instance to instead store this data in an S Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), new instances can configure only the - [`sourcegraph` bucket](#sourcegraph-bucket), and Sourcegraph will use that + [Sourcegraph bucket](#sourcegraph-bucket), and Sourcegraph will use that single bucket for all features. If a separate bucket is needed for Code Graph Indexes or Search Job Results, that can still be configured, but we recommend using one bucket. - - -## `sourcegraph` bucket +## Sourcegraph bucket Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), self-hosted Sourcegraph instances using S3 or - GCS object storage should now provision an additional bucket named - [`sourcegraph`](#sourcegraph-bucket). Sourcegraph currently reports a warning - when this bucket is not present, and it will become required for new features - in a future release. No action is required if you are using the default - `sourcegraph/blobstore`. + GCS object storage should now provision an additional bucket for shared + Sourcegraph uploads. Sourcegraph + currently reports a warning when this bucket is not present, and it will + become required for new features in a future release. No action is required + if you are using the default `sourcegraph/blobstore`. -The `sourcegraph` bucket is intended to be the single bucket for new Sourcegraph features. Instead of creating one bucket per feature, new features store objects under namespaced key prefixes within this bucket. +The Sourcegraph bucket is intended to be the single bucket for new Sourcegraph features. Instead of creating one bucket per feature, new features store objects under namespaced key prefixes within this bucket. Existing buckets for code graph indexes and search jobs remain in use. This change ensures future features can be enabled without requiring a new bucket for each feature. -### Using S3 for the `sourcegraph` bucket +### Using S3 for the Sourcegraph bucket Set the following environment variables to target an S3 bucket for shared Sourcegraph uploads. - `SOURCEGRAPH_UPLOAD_BACKEND=S3` -- `SOURCEGRAPH_UPLOAD_BUCKET=sourcegraph` (default) +- `SOURCEGRAPH_UPLOAD_BUCKET=` - `SOURCEGRAPH_UPLOAD_AWS_ENDPOINT=https://s3.us-east-1.amazonaws.com` - `SOURCEGRAPH_UPLOAD_AWS_ACCESS_KEY_ID=` - `SOURCEGRAPH_UPLOAD_AWS_SECRET_ACCESS_KEY=` - `SOURCEGRAPH_UPLOAD_AWS_SESSION_TOKEN=` (optional) - `SOURCEGRAPH_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) - `SOURCEGRAPH_UPLOAD_AWS_USE_PATH_STYLE=false` (optional) -- `SOURCEGRAPH_UPLOAD_AWS_REGION=us-east-1` (default) +- `SOURCEGRAPH_UPLOAD_AWS_REGION=` -### Using GCS for the `sourcegraph` bucket +### Using GCS for the Sourcegraph bucket Set the following environment variables to target a GCS bucket for shared Sourcegraph uploads. - `SOURCEGRAPH_UPLOAD_BACKEND=GCS` -- `SOURCEGRAPH_UPLOAD_BUCKET=sourcegraph` (default) +- `SOURCEGRAPH_UPLOAD_BUCKET=` - `SOURCEGRAPH_UPLOAD_GCP_PROJECT_ID=` -- `SOURCEGRAPH_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` -- `SOURCEGRAPH_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` +- `SOURCEGRAPH_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` (optional) +- `SOURCEGRAPH_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` (optional) + +If you are running on GKE with Workload Identity, or otherwise relying on +Application Default Credentials, you can omit the GCS credentials file +variables. -### Automatically provision the `sourcegraph` bucket +### Automatically provision the Sourcegraph bucket -If you would like to allow your Sourcegraph instance to manage the target bucket configuration, set the following environment variable: +Most deployments should provision this bucket directly in their cloud provider and leave this disabled. If you would like to allow your Sourcegraph instance to manage the target bucket configuration, set the following environment variable: This requires additional bucket-management permissions from your configured @@ -70,7 +72,7 @@ To target a managed object storage service for storing [code graph index uploads Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), new instances can configure only the - [`sourcegraph` bucket](#sourcegraph-bucket), and Sourcegraph will use that + [Sourcegraph bucket](#sourcegraph-bucket), and Sourcegraph will use that single bucket for all features. If a separate bucket is needed for Code Graph Indexes, that can still be configured, but we recommend using one bucket. @@ -94,7 +96,7 @@ To target an S3 bucket you've already provisioned, set the following environment - `PRECISE_CODE_INTEL_UPLOAD_AWS_SECRET_ACCESS_KEY=` - `PRECISE_CODE_INTEL_UPLOAD_AWS_SESSION_TOKEN=` (optional) - `PRECISE_CODE_INTEL_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) -- `PRECISE_CODE_INTEL_UPLOAD_AWS_REGION=us-east-1` (default) +- `PRECISE_CODE_INTEL_UPLOAD_AWS_REGION=` If a non-default region is supplied, ensure that the subdomain of the @@ -112,13 +114,17 @@ To target an S3 bucket you've already provisioned, set the following environment ### Using GCS for the Code Graph Indexes bucket -To target a GCS bucket you've already provisioned, set the following environment variables. Authentication is done through a service account key, supplied as either a path to a volume-mounted file, or the contents read in as an environment variable payload. +To target a GCS bucket you've already provisioned, set the following environment variables. - `PRECISE_CODE_INTEL_UPLOAD_BACKEND=GCS` - `PRECISE_CODE_INTEL_UPLOAD_BUCKET=` - `PRECISE_CODE_INTEL_UPLOAD_GCP_PROJECT_ID=` -- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` -- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` +- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` (optional) +- `PRECISE_CODE_INTEL_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` (optional) + +If you are running on GKE with Workload Identity, or otherwise relying on +Application Default Credentials, you can omit the GCS credentials file +variables. ### Automatically provision the Code Graph Indexes bucket @@ -138,7 +144,7 @@ To target a third party managed object storage service for storing [search job r Starting in [Sourcegraph 7.2](https://sourcegraph.com/changelog/releases/7.2), new instances can configure only the - [`sourcegraph` bucket](#sourcegraph-bucket), and Sourcegraph will use that + [Sourcegraph bucket](#sourcegraph-bucket), and Sourcegraph will use that single bucket for all features. If a separate bucket is needed for Search Job Results, that can still be configured, but we recommend using one bucket. @@ -162,7 +168,7 @@ Set the following environment variables to target an S3 bucket you've already pr - `SEARCH_JOBS_UPLOAD_AWS_SECRET_ACCESS_KEY=` - `SEARCH_JOBS_UPLOAD_AWS_SESSION_TOKEN=` (optional) - `SEARCH_JOBS_UPLOAD_AWS_USE_EC2_ROLE_CREDENTIALS=true` (optional; set to use EC2 metadata API over static credentials) -- `SEARCH_JOBS_UPLOAD_AWS_REGION=us-east-1` (default) +- `SEARCH_JOBS_UPLOAD_AWS_REGION=` If a non-default region is supplied, ensure that the subdomain of the @@ -177,13 +183,17 @@ Set the following environment variables to target an S3 bucket you've already pr ### Using GCS for the Search Job Results bucket -Set the following environment variables to target a GCS bucket you've already provisioned. Authentication is done through a service account key, either as a path to a volume-mounted file or the contents read in as an environment variable payload. +Set the following environment variables to target a GCS bucket you've already provisioned. - `SEARCH_JOBS_UPLOAD_BACKEND=GCS` - `SEARCH_JOBS_UPLOAD_BUCKET=` - `SEARCH_JOBS_UPLOAD_GCP_PROJECT_ID=` -- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` -- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` +- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE=` (optional) +- `SEARCH_JOBS_UPLOAD_GOOGLE_APPLICATION_CREDENTIALS_FILE_CONTENT=<{"my": "content"}>` (optional) + +If you are running on GKE with Workload Identity, or otherwise relying on +Application Default Credentials, you can omit the GCS credentials file +variables. ### Automatically provision the Search Job Results bucket