Tiny DeepSeek 1.5B Models Run on $99 NVIDIA Jetson Nano

Youtuber Ominous Industries recently demonstrated the capabilities of the DeepSeek R1 1.5B models running locally on the affordable NVIDIA Jetson Nano. The newly released distilled DeepSeek models were put to the test, showcasing their impressive performance even on the compact Jetson Nano hardware.

Installation and Testing

The video walkthrough begins with the installation process, followed by a series of performance tests. One notable test involved a Python reasoning challenge, where the DeepSeek R1 distilled 1.5B models were compared to the Mini Llama 3.1 1B model. The FP16 version of the DeepSeek R1 Distilled 1.5B Qwen model was also set up and tested, running smoothly in WebUI with an Ollama backend.

NVIDIA Jetson Nano Specifications

Here’s a breakdown of the NVIDIA Jetson Nano’s hardware and features:

• GPU: 128-core NVIDIA Maxwell architecture GPU
• CPU: Quad-core ARM Cortex-A57 MPCore processor
• Memory:
• Jetson Nano 4GB Developer Kit: 4GB 64-bit LPDDR4, 25.6 GB/s, 1600MHz
• Jetson Nano 2GB Developer Kit: 2GB 64-bit LPDDR4, 25.6 GB/s, 1600MHz (for the 2GB version)
• Storage: MicroSD card slot for OS and storage
• Video:
• 1x HDMI 2.0, 1x DisplayPort 1.3 over USB-C
• Supports resolutions up to 4K 60Hz
• Camera: 1x MIPI CSI-2 D-PHY interface, supports 2 cameras (in newer versions like V3)
• Connectivity:
• Gigabit Ethernet, 4x USB 3.0 ports
• 40-pin expansion header with GPIO pins
• Power:
• 5W to 10W power consumption; can be powered via micro-USB (4GB version) or USB-C (2GB version)
• Performance: Up to 472 GFLOPS of compute performance (FP16)
• Software: Supported by NVIDIA JetPack SDK, which includes CUDA, cuDNN, and TensorRT for AI and machine learning applications.

Cost

• NVIDIA Jetson Nano 4GB Developer Kit: $99
• NVIDIA Jetson Nano 2GB Developer Kit: $59

Key Takeaways

The DeepSeek R1 1.5B models, despite their compact size, deliver remarkable performance on the NVIDIA Jetson Nano. This combination of affordability and efficiency makes it an excellent choice for developers and hobbyists exploring AI and machine learning applications.