Amazon EC2 GPU Updated 2025-07-16
As of December 2023, the cheapest instance with an Nvidia GPU is g4nd.xlarge, so let's try that out. In that instance, lspci contains:so we see that it runs a Nvidia T4 GPU.
00:1e.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)Be careful not to confuse it with g4ad.xlarge, which has an AMD GPU instead. TODO meaning of "ad"? "a" presumably means AMD, but what is the "d"?
Some documentation on which GPU is in each instance can seen at: docs.aws.amazon.com/dlami/latest/devguide/gpu.html (archive) with a list of which GPUs they have at that random point in time. Can the GPU ever change for a given instance name? Likely not. Also as of December 2023 the list is already outdated, e.g. P5 is now shown, though it is mentioned at: aws.amazon.com/ec2/instance-types/p5/
When selecting the instance to launch, the GPU does not show anywhere apparently on the instance information page, it is so bad!
Also note that this instance has 4 vCPUs, so on a new account you must first make a customer support request to Amazon to increase your limit from the default of 0 to 4, see also: stackoverflow.com/questions/68347900/you-have-requested-more-vcpu-capacity-than-your-current-vcpu-limit-of-0, otherwise instance launch will fail with:
You have requested more vCPU capacity than your current vCPU limit of 0 allows for the instance bucket that the specified instance type belongs to. Please visit aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit.
When starting up the instance, also select:Once you finally managed to SSH into the instance, first we have to install drivers and reboot:and now running:shows something like:
- image: Ubuntu 22.04
- storage size: 30 GB (maximum free tier allowance)
sudo apt update
sudo apt install nvidia-driver-510 nvidia-utils-510 nvidia-cuda-toolkit
sudo rebootnvidia-smi+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:1E.0 Off | 0 |
| N/A 25C P8 12W / 70W | 2MiB / 15360MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+If we start from the raw Ubuntu 22.04, first we have to install drivers:
- docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html official docs
- stackoverflow.com/questions/63689325/how-to-activate-the-use-of-a-gpu-on-aws-ec2-instance
- askubuntu.com/questions/1109662/how-do-i-install-cuda-on-an-ec2-ubuntu-18-04-instance
- askubuntu.com/questions/1397934/how-to-install-nvidia-cuda-driver-on-aws-ec2-instance
From there basically everything should just work as normal. E.g. we were able to run a CUDA hello world just fine along:
nvcc inc.cu
./a.outOne issue with this setup, besides the time it takes to setup, is that you might also have to pay some network charges as it downloads a bunch of stuff into the instance. We should try out some of the pre-built images. But it is also good to know this pristine setup just in case.
We then managed to run Ollama just fine with:which gave:so way faster than on my local desktop CPU, hurray.
curl https://ollama.ai/install.sh | sh
/bin/time ollama run llama2 'What is quantum field theory?'0.07user 0.05system 0:16.91elapsed 0%CPU (0avgtext+0avgdata 16896maxresident)k
0inputs+0outputs (0major+1960minor)pagefaults 0swapsAfter setup from: askubuntu.com/a/1309774/52975 we were able to run:which gave:so only marginally better than on P14s. It would be fun to see how much faster we could make things on a more powerful GPU.
head -n1000 pap.txt | ARGOS_DEVICE_TYPE=cuda time argos-translate --from-lang en --to-lang fr > pap-fr.txt77.95user 2.87system 0:39.93elapsed 202%CPU (0avgtext+0avgdata 4345988maxresident)k
0inputs+88outputs (0major+910748minor)pagefaults 0swaps Amazon EC2 hello world Updated 2025-07-16
As of December 2023 on a
t2.micro instance, the only one part of free tier at the time with advertised 1 vCPU, 1 GiB RAM, 8 GiB disk for the first 12 months, on Ubuntu 22.04:$ free -h
total used free shared buff/cache available
Mem: 949Mi 149Mi 210Mi 0.0Ki 590Mi 641Mi
Swap: 0B 0B 0B
$ nproc
1
$ df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/root 7.6G 1.8G 5.8G 24% /To install software:
sudo apt update
sudo apt install cowsay
cowsay asdf Ubuntu 18.04 Updated 2025-07-16
Ubuntu 23.10 Updated 2025-07-16
Virtuoso Universal Server Updated 2025-07-16
RDFlib Updated 2025-07-16
Apache Jena Updated 2025-07-16
RDNA 3 Updated 2025-07-16
qiskit/qft.py Updated 2025-07-16
This is an example of the
qiskit.circuit.library.QFT implementation of the Quantum Fourier transform function which is documented at: docs.quantum.ibm.com/api/qiskit/0.44/qiskit.circuit.library.QFTOutput:So this also serves as a more interesting example of quantum compilation, mapping the
init: [1, 0, 0, 0, 0, 0, 0, 0]
qc
┌──────────────────────────────┐┌──────┐
q_0: ┤0 ├┤0 ├
│ ││ │
q_1: ┤1 Initialize(1,0,0,0,0,0,0,0) ├┤1 QFT ├
│ ││ │
q_2: ┤2 ├┤2 ├
└──────────────────────────────┘└──────┘
transpiled qc
┌──────────────────────────────┐ ┌───┐
q_0: ┤0 ├────────────────────■────────■───────┤ H ├─X─
│ │ ┌───┐ │ │P(π/2) └───┘ │
q_1: ┤1 Initialize(1,0,0,0,0,0,0,0) ├──────■───────┤ H ├─┼────────■─────────────┼─
│ │┌───┐ │P(π/2) └───┘ │P(π/4) │
q_2: ┤2 ├┤ H ├─■─────────────■──────────────────────X─
└──────────────────────────────┘└───┘
Statevector([0.35355339+0.j, 0.35355339+0.j, 0.35355339+0.j,
0.35355339+0.j, 0.35355339+0.j, 0.35355339+0.j,
0.35355339+0.j, 0.35355339+0.j],
dims=(2, 2, 2))
init: [0.0, 0.35355339059327373, 0.5, 0.3535533905932738, 6.123233995736766e-17, -0.35355339059327373, -0.5, -0.35355339059327384]
Statevector([ 7.71600526e-17+5.22650714e-17j,
1.86749130e-16+7.07106781e-01j,
-6.10667421e-18+6.10667421e-18j,
1.13711443e-16-1.11022302e-16j,
2.16489014e-17-8.96726857e-18j,
-5.68557215e-17-1.11022302e-16j,
-6.10667421e-18-4.94044770e-17j,
-3.30200457e-16-7.07106781e-01j],
dims=(2, 2, 2))QFT gate to Qiskit Aer primitives.If we don't
transpile in this example, then running blows up with:qiskit_aer.aererror.AerError: 'unknown instruction: QFT'The second input is:and the output of that approximately:which can be defined simply as the normalized DFT of the input quantum state vector.
[0, 1j/sqrt(2), 0, 0, 0, 0, 0, 1j/sqrt(2)]From this we see that the Quantum Fourier transform is equivalent to a direct discrete Fourier transform on the quantum state vector, related: physics.stackexchange.com/questions/110073/how-to-derive-quantum-fourier-transform-from-discrete-fourier-transform-dft
qiskit/initialize.py Updated 2025-07-16
In this example we will initialize a quantum circuit with a single CNOT gate and see the output values.
By default, Qiskit initializes every qubit to 0 as shown in the qiskit/hello.py. But we can also initialize to arbitrary values as would be done when computing the output for various different inputs.
Output:which we should all be able to understand intuitively given our understanding of the CNOT gate and quantum state vectors.
┌──────────────────────┐
q_0: ┤0 ├──■──
│ Initialize(1,0,0,0) │┌─┴─┐
q_1: ┤1 ├┤ X ├
└──────────────────────┘└───┘
c: 2/═════════════════════════════
init: [1, 0, 0, 0]
probs: [1. 0. 0. 0.]
init: [0, 1, 0, 0]
probs: [0. 0. 0. 1.]
init: [0, 0, 1, 0]
probs: [0. 0. 1. 0.]
init: [0, 0, 0, 1]
probs: [0. 1. 0. 0.]
┌──────────────────────────────────┐
q_0: ┤0 ├──■──
│ Initialize(0.70711,0,0,0.70711) │┌─┴─┐
q_1: ┤1 ├┤ X ├
└──────────────────────────────────┘└───┘
c: 2/═════════════════════════════════════════
init: [0.7071067811865475, 0, 0, 0.7071067811865475]
probs: [0.5 0.5 0. 0. ]quantumcomputing.stackexchange.com/questions/13202/qiskit-initializing-n-qubits-with-binary-values-0s-and-1s describes how to initialize circuits qubits only with binary 0 or 1 to avoid dealing with the exponential number of elements of the quantum state vector.
qiskit.transpile() Updated 2025-07-16 City of Light: The Story of Fiber Optics Updated 2025-07-16
DeepFloyd IF Updated 2025-07-16
runwayml/stable-diffusion Updated 2025-07-16
Someone should package this better for end user "just works after Conda install" image generation, it is currently much more of a library setup.
Tested on Amazon EC2 on a g5.xlarge machine, which has an Nvidia A10G, using the AWS Deep Learning Base GPU AMI (Ubuntu 20.04) image.
First install Conda as per Section "Install Conda on Ubuntu", and then just follow the instructions from the README, notably the Reference sampling script section.This took about 2 minutes and generated 6 images under
git clone https://github.com/runwayml/stable-diffusion
cd stable-diffusion/
git checkout 08ab4d326c96854026c4eb3454cd3b02109ee982
conda env create -f environment.yaml
conda activate ldm
mkdir -p models/ldm/stable-diffusion-v1/
wget -O models/ldm/stable-diffusion-v1/model.ckpt https://huggingface.co/CompVis/stable-diffusion-v-1-4-original/resolve/main/sd-v1-4.ckpt
python scripts/txt2img.py --prompt "a photograph of an astronaut riding a horse" --plmsoutputs/txt2img-samples/samples, includining an image outputs/txt2img-samples/grid-0000.png which is a grid montage containing all the six images in one:
A quick attempt at removing their useless safety features (watermark and NSFW text filter) is:but that produced 4 black images and only two unfiltered ones. Also likely the lack of sexual training data makes its porn suck, and not in the good way.
diff --git a/scripts/txt2img.py b/scripts/txt2img.py
index 59c16a1..0b8ef25 100644
--- a/scripts/txt2img.py
+++ b/scripts/txt2img.py
@@ -87,10 +87,10 @@ def load_replacement(x):
def check_safety(x_image):
safety_checker_input = safety_feature_extractor(numpy_to_pil(x_image), return_tensors="pt")
x_checked_image, has_nsfw_concept = safety_checker(images=x_image, clip_input=safety_checker_input.pixel_values)
- assert x_checked_image.shape[0] == len(has_nsfw_concept)
- for i in range(len(has_nsfw_concept)):
- if has_nsfw_concept[i]:
- x_checked_image[i] = load_replacement(x_checked_image[i])
+ #assert x_checked_image.shape[0] == len(has_nsfw_concept)
+ #for i in range(len(has_nsfw_concept)):
+ # if has_nsfw_concept[i]:
+ # x_checked_image[i] = load_replacement(x_checked_image[i])
return x_checked_image, has_nsfw_concept
@@ -314,7 +314,7 @@ def main():
for x_sample in x_checked_image_torch:
x_sample = 255. * rearrange(x_sample.cpu().numpy(), 'c h w -> h w c')
img = Image.fromarray(x_sample.astype(np.uint8))
- img = put_watermark(img, wm_encoder)
+ # img = put_watermark(img, wm_encoder)
img.save(os.path.join(sample_path, f"{base_count:05}.png"))
base_count += 1 ludicrains/deep-gaze Updated 2025-07-16
This just works, but it is also so incredibly slow that it is useless (or at least the quality it reaches in the time we have patience to wait from), at least on any setup we've managed to try, including e.g. on an Nvidia A10G on a g5.xlarge. Running:would likely take hours to complete.
time imagine "a house in the forest" OpenNMT Updated 2025-07-16
Open source LLM Updated 2025-07-19
GitHub repo Updated 2025-07-16
bsub get job stdout and stderr Updated 2025-07-16
By default, LSF only sends you an email with the stdout and stderr included in it, and does not show or store anything locally.
One option to store things locally is to use:as documented at:
bsub -oo stdout.log -eo stderr.log 'echo myout; echo myerr 1>&2'Or to use files with the job id in them:
bsub -oo %J.out -eo %J.err 'echo myout; echo myerr 1>&2'By default as mentioned at:
bsub -oo:To get just the stdout to the file, use
bsub -N -oo which:- stores only stdout on the file
- re-enables the completion email
Another option is to run with the bsub This immediately prints stdout and stderr to the terminal.
-I option:bsub -I 'echo a;sleep 1;echo b;sleep 1;echo c' AMDGPU Updated 2025-07-16
Bibliography:
There are unlisted articles, also show them or only show them.