Secure, Reproducible AWS EC2 Ubuntu GPU Setup for Deep Learning (2026)

Table of Contents

A secure, reproducible Ubuntu GPU workflow on Amazon EC2

The 2017 checklist mixed a world-writable Git deployment hook, an obsolete Bazel repository setup, an unspecified TensorFlow source build, unpinned Python packages, and a public-notebook tutorial. None of those choices is suitable as a maintained default.

This 2026 edition starts with a version matrix and a cost boundary, keeps Jupyter on loopback behind an authenticated tunnel, and records enough evidence to reproduce or retire the machine safely. It is a workflow, not a promise that every framework release works with every GPU, CPU architecture, driver, or CUDA stack. Check the selected release’s official matrix before launch and again before installation.

1. Choose one platform path deliberately

Path	Best fit	Reproducibility boundary
AWS Deep Learning AMI (DLAMI)	Fastest supported start for interactive work	Pin Region and AMI ID, read that image’s release notes, activate its documented environment, and do not casually replace preinstalled CUDA or framework packages
AWS Deep Learning Container	Team workflows, CI, and repeatable jobs	Pin the image by immutable digest, record the host driver and GPU runtime, and keep data and secrets outside the image
Standard Ubuntu AMI	Maximum control or a requirement not met by AWS images	Pin the Ubuntu AMI and install the driver/toolkit only through the current NVIDIA instructions; this path carries the most compatibility and patching work

AWS DLAMIs arrive with common framework and NVIDIA components configured. AWS Deep Learning Container documentation points to the maintained container catalog. A standard Ubuntu image is not automatically more reproducible: it is reproducible only when the AMI, repository keyring, package versions, framework lock, and verification output are recorded.

Do not mix paths mid-install. In particular, do not layer an unrelated CUDA runfile, distribution packages, and pip-provided CUDA libraries into one environment unless the selected framework’s documentation explicitly supports that combination.

2. Build the compatibility inventory before launch

Start from the framework release you need, then work outward—not from the newest-looking GPU. Record every cell below:

Layer	Exact value to record	Authority/check
AWS placement	Region, Availability Zone, instance type, purchase model	Current EC2 console/API and the accelerated-instance specification
Machine image	AMI ID, image name, owner, creation date	EC2 image details; IDs are Region-specific
CPU	`x86_64` or `arm64`	Instance specification plus `uname -m`
GPU	Model, count, memory, NVIDIA architecture, compute capability	EC2 specification, `nvidia-smi`, and the NVIDIA CUDA GPU table
Driver	Exact driver version	`nvidia-smi` and the NVIDIA driver utility reference
CUDA	Driver-supported CUDA ceiling; separately, installed toolkit version	`nvidia-smi` is not proof that `nvcc` is installed; use `nvcc --version` when applicable
Framework	Name, exact version/build, supported Python/CUDA/cuDNN/compute capability	The selected framework release’s official install/build matrix
Python environment	Python, pip, every direct and transitive package, wheel hashes	Lock file created for this OS, CPU architecture, and Python version

Stop before launch if a cell is unknown, the required AMI is not owned by the expected publisher, the CPU architecture lacks the required wheels, the GPU has insufficient memory or compute capability, or the framework matrix contradicts the chosen driver/CUDA combination.

3. Put security and cost controls in place first

Create an AWS Budget and alert before starting a GPU instance. An alert is notification, not an automatic shutdown; configure and test a budget action separately if that is the intended control. Review current rates, Spot interruption behavior, quotas, EBS, snapshots, public IPv4 addresses, data transfer, and idle resources in the chosen Region.
Prefer Systems Manager Session Manager with no inbound rule. If SSH is required, allow TCP 22 only from a fixed administrator CIDR. Never open 8888 to 0.0.0.0/0 or ::/0.
Attach a least-privilege instance role for only the required artifact, data, log, and Systems Manager resources. Do not copy long-lived AWS access keys onto the instance.
Require IMDSv2. Keep the response hop limit at 1 for a host-only workflow; container networking may require a deliberately reviewed value. Do not expose instance tags through metadata unless required.
Encrypt EBS volumes, separate durable data from disposable workspace where practical, and inspect each volume’s DeleteOnTermination value. Treat instance-store data as temporary.
Use a dedicated Unix user and restrictive permissions. Do not use chmod -R 777, passwordless deployment hooks, or a shared writable checkout.

From an authorized administration machine, require IMDSv2 for an existing instance:

INSTANCE_ID='replace-with-instance-id'

aws ec2 modify-instance-metadata-options \
  --instance-id "$INSTANCE_ID" \
  --http-tokens required \
  --http-endpoint enabled \
  --http-put-response-hop-limit 1 \
  --instance-metadata-tags disabled

The relevant first-party references are IMDS configuration, EC2 IAM roles, security-group rules, EBS encryption, and AWS Budgets.

4. Inventory the running host before changing it

Run this as the normal instance user. The IMDSv2 token is short-lived, is not written to the manifest, and is unset after the two non-secret metadata fields are captured. The header is fed to curl through standard input rather than its argument list; still treat the shell and host as trusted. Keep run-manifest/ private because operational inventories can reveal sensitive infrastructure details.

set -euo pipefail
umask 077
mkdir -p run-manifest

cat /etc/os-release | tee run-manifest/os-release.txt
uname -m | tee run-manifest/cpu-architecture.txt
python3 --version 2>&1 | tee run-manifest/python-system.txt

metadata_base='http://169.254.169.254/latest'
metadata_token="$(curl --fail --silent --show-error \
  --request PUT \
  --header 'X-aws-ec2-metadata-token-ttl-seconds: 60' \
  "$metadata_base/api/token")"
for key in ami-id instance-type; do
  builtin printf \
    'header = "X-aws-ec2-metadata-token: %s"\n' \
    "$metadata_token" \
    | curl --fail --silent --show-error \
      --config - \
      "$metadata_base/meta-data/$key" \
    | tee "run-manifest/$key.txt"
  printf '\n'
done
unset metadata_token

command -v nvidia-smi >/dev/null || {
  printf '%s\n' 'nvidia-smi is missing; stop and repair the driver path.' >&2
  exit 1
}
nvidia-smi | tee run-manifest/nvidia-smi.txt
nvidia-smi \
  --query-gpu=name,pci.bus_id,driver_version,memory.total \
  --format=csv,noheader \
  | tee run-manifest/gpu-inventory.csv

if command -v nvcc >/dev/null; then
  nvcc --version | tee run-manifest/nvcc.txt
else
  printf '%s\n' 'nvcc not installed; no local CUDA toolkit recorded.' \
    | tee run-manifest/nvcc.txt
fi

On a standard Ubuntu path, follow the current NVIDIA CUDA Installation Guide for Linux. Use its supported repository/keyring flow and pre-install checks; do not revive the old apt-key, unsigned repository, or HTTP package-source commands. After any driver change, reboot if the official instructions require it, rerun the inventory, and stop on a driver/library mismatch.

5. Create an isolated, locked Python environment

Use the Python version supported by the chosen framework build. A virtual environment isolates Python packages, but it does not solve driver or system-library compatibility.

Create requirements.lock on the same OS, CPU architecture, Python minor version, and package index policy used for production. Pin every direct and transitive dependency and include verified hashes. Review the lock before use; do not type an unversioned pip install tensorflow numpy pandas jupyter into a long-lived environment.

set -euo pipefail
umask 077

python3 -m venv .venv
.venv/bin/python -m pip install \
  --require-hashes \
  --only-binary=:all: \
  --requirement requirements.lock
.venv/bin/python -m pip check
.venv/bin/python -m pip freeze --all \
  | tee run-manifest/python-freeze.txt
sha256sum requirements.lock \
  | tee run-manifest/requirements-lock.sha256

For a standard supported Linux GPU installation, TensorFlow’s current official pip path uses the tensorflow[and-cuda] extra, but the exact release must still be pinned with all resolved dependencies in the lock. A DLAMI may instead require activation of its documented prebuilt environment. Consult TensorFlow’s pip installation guide, Python’s venv documentation, and pip’s repeatable-install guidance.

6. Prove that the framework executes on the GPU

Run a framework-level operation, not only nvidia-smi. The following TensorFlow smoke test reports the package build, visible devices, and device details, then requires a matrix multiplication to be placed on GPU 0:

import json
import platform

import tensorflow as tf

tf.random.set_seed(20260901)
gpus = tf.config.list_physical_devices("GPU")
report = {
    "python": platform.python_version(),
    "tensorflow": tf.__version__,
    "build": tf.sysconfig.get_build_info(),
    "gpus": [
        tf.config.experimental.get_device_details(device)
        for device in gpus
    ],
}
print(json.dumps(report, indent=2, default=str))

if not gpus:
    raise SystemExit("no GPU visible to TensorFlow")

with tf.device("/GPU:0"):
    left = tf.random.uniform((512, 512))
    right = tf.random.uniform((512, 512))
    product = tf.linalg.matmul(left, right)

print({"device": product.device, "shape": product.shape.as_list()})
if "GPU:0" not in product.device.upper():
    raise SystemExit(f"operation was not placed on GPU 0: {product.device}")

Run it with the locked interpreter and retain the output with the job record. A GPU being visible does not prove numerical correctness, model convergence, multi-GPU communication, mixed-precision behavior, or memory capacity; add a small representative test for the actual workload.

7. Keep Jupyter local and authenticated

Jupyter Server grants code execution, so treat access as shell access. Its token authentication is enabled by default. Leave that authentication enabled, keep the generated token out of tickets and logs, and bind only to loopback:

set -euo pipefail
umask 077
mkdir -p notebooks
cd notebooks

../.venv/bin/jupyter lab \
  --no-browser \
  --ServerApp.ip=127.0.0.1 \
  --ServerApp.port=8888 \
  --ServerApp.port_retries=0 \
  --ServerApp.open_browser=False

Do not add an inbound security-group rule for 8888. From the administrator’s machine, use one of these tunnels. For SSH:

ssh -N \
  -L 127.0.0.1:8888:127.0.0.1:8888 \
  ubuntu@replace-with-hostname

Or, after completing Session Manager prerequisites and installing its local plugin:

INSTANCE_ID='replace-with-managed-instance-id'

aws ssm start-session \
  --target "$INSTANCE_ID" \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["8888"],"localPortNumber":["8888"]}'

Open http://127.0.0.1:8888/ locally and enter the token shown only in the instance terminal. If persistent password authentication is required, use Jupyter’s interactive password command so only a hash is stored. Never disable both password and token. See Jupyter Server security, its loopback default and public-server warning, and AWS Session Manager port forwarding.

8. Keep code, secrets, data, and logs separate

Code: use a normal user-owned checkout. Record the exact commit and submodules with Git; do not deploy through an unreviewed post-receive hook.
Secrets: use a scoped instance role and an approved secret store. Never commit keys, place them in notebooks, bake them into an AMI/container, or print them in inventory logs.
Input data: mount or download read-only where possible. Record dataset version and checksums, not private object URLs or customer identifiers.
Outputs: write checkpoints to a durable encrypted EBS volume or approved object storage at explicit intervals. Verify a remote copy before stopping an instance that uses instance store.
Logs: capture package, driver, framework, command, exit status, timestamps, and workload metrics; redact tokens, signed URLs, prompts, and personal or regulated data before centralizing them.

Capture source state without changing deployment permissions:

set -euo pipefail

git status --porcelain=v1
git rev-parse HEAD | tee run-manifest/source-commit.txt
git submodule status --recursive \
  | tee run-manifest/source-submodules.txt

The commit command is documented by Git `rev-parse`. A dirty status is a reproducibility stop gate: commit, discard, or explicitly archive the diff before launching an expensive job.

9. Treat source builds as an exception

Use a released wheel, DLAMI environment, or immutable container unless a source patch, unsupported compute capability, or compiler requirement makes a source build necessary. Before building TensorFlow, fill this matrix from its tested build configurations and the relevant Bazel installation documentation:

Required pin	Recorded value
TensorFlow tag and full Git commit
Python version
Bazel version
Compiler and standard-library versions
NVIDIA driver, CUDA toolkit, and cuDNN
GPU model, architecture, and compute capability
Base AMI or container digest
Build flags, patches, wheel hash, and expected tests

Do not assume current Bazel builds an old TensorFlow release. Do not proceed while any compatibility cell is inferred rather than documented.

set -euo pipefail
SOURCE_REF='replace-with-reviewed-tag-or-commit'

git clone --filter=blob:none \
  https://github.com/tensorflow/tensorflow.git
git -C tensorflow fetch --tags --prune
git -C tensorflow checkout --detach "$SOURCE_REF"
git -C tensorflow status --porcelain=v1
git -C tensorflow rev-parse HEAD

Build in a disposable volume or container, save the full log, verify the wheel in a fresh environment outside the source tree, and retain its checksum. A source build that merely finishes is not evidence that the GPU path or target workload works.

10. Maintain a private reproducibility manifest

Keep a machine-readable manifest beside the lock, test output, and checksums. Do not publish account IDs, instance IDs, internal hostnames, bucket names, object paths, tokens, or dataset identifiers.

schema_version: 1
captured_at_utc: replace-with-iso-8601-time
aws:
  region: replace
  availability_zone: replace
  instance_type: replace
  purchase_model: replace
  ami_id: replace
  ami_name: replace
  cpu_architecture: replace
gpu:
  model: replace
  count: replace
  architecture: replace
  compute_capability: replace
  driver_version: replace
  driver_cuda_ceiling: replace
  toolkit_version: replace-or-not-installed
environment:
  python_version: replace
  framework_name: tensorflow
  framework_version: replace
  framework_build: replace
  requirements_lock_sha256: replace
source:
  commit: replace
  dirty: false
  container_digest: replace-or-not-used
verification:
  gpu_smoke_test: pass-or-fail
  workload_smoke_test: pass-or-fail
storage:
  root_delete_on_termination: replace
  durable_output_location: private-reference-only

The manifest contains system facts, not credentials. Store sensitive resource mappings separately under narrower access control.

11. Verify, stop, roll back, and clean up

Before paid work:

Confirm the budget, alert recipients, instance price model, and a named person or automation responsible for stopping idle capacity.
Confirm the security group has no notebook port, IMDSv2 is required, the instance role is minimal, volumes are encrypted, and SSH/SSM access is logged as required.
Reconcile EC2 model/architecture with uname -m, GPU inventory, compute capability, driver, CUDA toolkit, locked Python packages, and framework build metadata.
Run pip check, the GPU smoke test, a small workload test, checkpoint restore, and tunnel-only Jupyter access.
Confirm logs and manifests contain no secrets or private data and that durable output can be restored independently of the instance.

When work ends, flush and verify checkpoints, stop Jupyter, copy the manifest and required logs, then stop the instance from an authorized administration machine:

INSTANCE_ID='replace-with-instance-id'

aws ec2 stop-instances --instance-ids "$INSTANCE_ID"
aws ec2 wait instance-stopped --instance-ids "$INSTANCE_ID"

Stopping ends instance compute charges but does not remove every cost: EBS, snapshots, public IPv4/Elastic IP resources, and other services can continue billing. Stop/start erases instance-store data. Termination deletes the root EBS volume by default, while other EBS volumes depend on DeleteOnTermination; review EC2 instance state changes before either action.

Terminate only after an explicit human gate verifies backups, restoration, volume flags, shared dependencies, and retention obligations. For rollback, keep the immutable AMI/container reference, environment lock, source commit, manifest, data version, and known-good test outputs; rebuild from them instead of repairing an undocumented machine in place.

12. Official references

13. Archived 2017 source

The complete visible body from source_export follows for provenance. Only trailing whitespace has been normalized; no content has been added, removed, corrected, or activated. The obsolete HTTP repository, apt-key, world-writable hook, unpinned install list, and public-Jupyter link are historical evidence and must not be executed.


1. Git server building.`

$ ssh-keygen -t rsa -C "user.email"`
 Modifying hooks:`

$ vim sample.git/hooks/post-receive`
 Sample code for hooks:`

#!/bin/sh

GIT_WORK_TREE=/home/ubuntu/Deployment/sample git checkout -f

chmod -R 777 /home/ubuntu/Deployment/sample`
2. Install bazel.
 1). Add Bazel distribution URI as a package source (one time setup)

echo “deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8” | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add –


If you want to install the testing version of Bazel, replace stable with testing.2). Install and update Bazel

sudo apt-get update && sudo apt-get install bazel
 Once installed, you can upgrade to a newer version of Bazel with:

sudo apt-get upgrade bazel


3. Tensorflow compiling.
4. Python module installing.

  - tensorflow
  - numpy
  - pandas

5. Juypter.
 https://punchagan.muse-amuse.in/posts/create-a-public-jupyter-server-quickly.html