By the end of this guide you have a running AI inference API on your own server no OpenAI key required.<\/p>\n\n\n\n
What You Will Build<\/h3>\n\n\n\n
You will deploy a complete local AI inference stack on a VPS:<\/p>\n\n\n\n
A Python environment with Hugging Face Transformers and PyTorch installed. A GPT-2 text generation model running locally. A Flask API server that accepts POST requests with a prompt and returns generated text. A live endpoint you can call from any application, n8n workflow , Telegram bot, or browser.<\/p>\n\n\n\n
This is the same foundational setup that teams use before adding GPU servers, larger models, and production load balancing. Start here, understand every layer, then scale.<\/p>\n\n\n\n
Step 1 – Choose the Right VPS<\/h3>\n\n\n\n
AI model inference requires more RAM than typical web hosting. The model weights load into memory when the server starts and stay there for the lifetime of the process. GPT-2, the model used in this guide, requires approximately 500MB of RAM. Larger models like DistilBERT need 1-2GB. Production-grade models like LLaMA variants need 8-16GB or more.<\/p>\n\n\n\n
For this beginner setup, choose a VPS with:<\/p>\n\n\n\n
- \n
- Minimum 4GB RAM (8GB recommended for comfort)<\/li>\n\n\n\n
- 4 vCPUs<\/li>\n\n\n\n
- 20GB+ SSD storage<\/li>\n\n\n\n
- Ubuntu 22.04 LTS<\/li>\n<\/ul>\n\n\n\n
Ubuntu 22.04 is the most stable choice for Python AI workloads in 2026. The package repositories are current, PyTorch has first-class support for it, and the Hugging Face documentation assumes it.<\/p>\n\n\n\n
A Ucartz VPS on the standard plan covers these requirements for AI inference workloads that do not need a dedicated GPU.<\/p>\n\n\n\n
Step 2 – Connect to Your VPS<\/h3>\n\n\n\n
From Windows PowerShell or any terminal:<\/p>\n\n\n\n
ssh root@YOUR_SERVER_IP<\/code><\/pre>\n\n\n\nAccept the fingerprint prompt by typing
yes<\/code>. Enter your password. You are now inside the server.<\/p>\n\n\n\nStep 3 – Update the Server<\/h3>\n\n\n\n
Always update packages before installing anything:<\/p>\n\n\n\n
apt update && apt upgrade -y<\/code><\/pre>\n\n\n\nThis ensures you are working with current package versions and avoids dependency conflicts during Python and PyTorch installation.<\/p>\n\n\n\n
Step 4 – Install Python and Pip<\/h3>\n\n\n\n
Ubuntu 22.04 ships with Python 3.10. Install it along with pip:<\/p>\n\n\n\n
apt install python3 python3-pip python3-venv -y<\/code><\/pre>\n\n\n\nVerify the installation:<\/p>\n\n\n\n
python3 --version\npip3 --version<\/code><\/pre>\n\n\n\nYou need Python 3.8 or higher for current Hugging Face Transformers. Python 3.10 on Ubuntu 22.04 works cleanly with all dependencies in this guide.<\/p>\n\n\n\n
Step 5 – Create Project Directory<\/h3>\n\n\n\n
mkdir ai-model\ncd ai-model<\/code><\/pre>\n\n\n\nKeep all project files inside this directory. When you expand to multiple models later, you create separate directories for each one keeping environments and dependencies isolated.<\/p>\n\n\n\n
![\"python](\"https:\/\/www.ucartz.com\/blog\/wp-content\/uploads\/2026\/05\/python-install-1024x316.webp\")
<\/figure>\n\n\n\nStep 6 – Create a Virtual Environment<\/h3>\n\n\n\n
A virtual environment isolates your project’s Python packages from the system Python installation. This prevents version conflicts between projects and makes your setup reproducible.<\/p>\n\n\n\n
python3 -m venv venv\nsource venv\/bin\/activate<\/code><\/pre>\n\n\n\nYour terminal prompt changes to show:<\/p>\n\n\n\n
(venv) root@your-server:~\/ai-model#<\/code><\/pre>\n\n\n\nEvery package you install from this point installs inside the virtual environment, not system-wide. Always activate the virtual environment before running your scripts.<\/p>\n\n\n\n
![\"venv](\"https:\/\/www.ucartz.com\/blog\/wp-content\/uploads\/2026\/05\/venv-environmment-1024x472.webp\")
<\/figure>\n\n\n\nStep 7 – Install Hugging Face Libraries<\/h3>\n\n\n\n
pip install transformers torch accelerate<\/code><\/pre>\n\n\n\nThis installs three packages.
transformers<\/code> is the Hugging Face library that provides model downloading, loading, and inference pipelines.torch<\/code> is PyTorch the deep learning framework that runs the actual model computations.accelerate<\/code> is a Hugging Face utility that optimizes model loading and inference on CPU and GPU.<\/p>\n\n\n\nThis installation takes 3-5 minutes. PyTorch alone is approximately 700MB. The download speed depends on your VPS provider’s bandwidth Ucartz VPS instances have unmetered bandwidth which helps here.<\/p>\n\n\n\n
Verify the installation:<\/p>\n\n\n\n
python3 -c \"import transformers; print(transformers.__version__)\"\npython3 -c \"import torch; print(torch.__version__)\"<\/code><\/pre>\n\n\n\nBoth commands should print version numbers without errors.<\/p>\n\n\n\n
Step 8 – Write Your First Inference Script<\/h3>\n\n\n\n
Create the script file:<\/p>\n\n\n\n
nano app.py<\/code><\/pre>\n\n\n\nPaste this code:<\/p>\n\n\n\n
from transformers import pipeline\n\n# Load the text generation pipeline with GPT-2\ngenerator = pipeline(\"text-generation\", model=\"gpt2\")\n\n# Run inference\nresult = generator(\"AI will change business because\", max_length=50)\n\nprint(result[0]['generated_text'])<\/code><\/pre>\n\n\n\nSave the file with CTRL+X, then Y, then ENTER.<\/p>\n\n\n\n
Step 9 – Run Your First Local AI Inference<\/h3>\n\n\n\n
python3 app.py<\/code><\/pre>\n\n\n\nThe first run downloads the GPT-2 model from Hugging Face. GPT-2 is approximately 500MB. The download speed depends on your VPS network. After the download completes, the model loads into memory and inference runs.<\/p>\n\n\n\n
Output appears like:<\/p>\n\n\n\n
AI will change business because it allows companies to process\ninformation faster than any human team and make decisions at a\nscale that was previously impossible without large...<\/code><\/pre>\n\n\n\nThe exact text varies GPT-2 is a generative model. What matters is that this text was generated entirely on your server. No API key. No external service. No per-token cost. You just ran local AI inference.<\/p>\n\n\n\n
The model download only happens once. Hugging Face caches models in
~\/.cache\/huggingface\/<\/code> after the first download. Every subsequent run loads the cached model from disk.<\/p>\n\n\n\n![\"hugging](\"https:\/\/www.ucartz.com\/blog\/wp-content\/uploads\/2026\/05\/hugging-face-ai-1024x483.webp\")
<\/figure>\n\n\n\nStep 10 – Build a Flask API Around the Model<\/h3>\n\n\n\n
Running a script manually is useful for testing. To integrate AI inference with n8n workflow automation, Telegram bots, web applications, or any other system, you need an HTTP API.<\/p>\n\n\n\n
Install Flask:<\/p>\n\n\n\n
pip install flask<\/code><\/pre>\n\n\n\nCreate the API server:<\/p>\n\n\n\n
nano api.py<\/code><\/pre>\n\n\n\nPaste this code:<\/p>\n\n\n\n
from flask import Flask, request, jsonify\nfrom transformers import pipeline\n\napp = Flask(__name__)\n\n# Load model once at startup \u2014 not on every request\nprint(\"Loading model...\")\ngenerator = pipeline(\"text-generation\", model=\"gpt2\")\nprint(\"Model loaded. API ready.\")\n\n@app.route('\/generate', methods=['POST'])\ndef generate():\n data = request.get_json()\n\n if not data or 'prompt' not in data:\n return jsonify({\"error\": \"Missing 'prompt' field\"}), 400\n\n prompt = data['prompt']\n max_length = data.get('max_length', 100)\n\n result = generator(prompt, max_length=max_length, num_return_sequences=1)\n\n return jsonify({\n \"prompt\": prompt,\n \"response\": result[0]['generated_text'],\n \"model\": \"gpt2\"\n })\n\n@app.route('\/health', methods=['GET'])\ndef health():\n return jsonify({\"status\": \"ok\", \"model\": \"gpt2\"})\n\nif __name__ == '__main__':\n app.run(host='0.0.0.0', port=5000)<\/code><\/pre>\n\n\n\nSave with CTRL+X, Y, ENTER.<\/p>\n\n\n\n
The model loads once when the server starts. If you reloaded the model on every request, each API call would take 30-60 seconds to respond. Loading at startup means inference requests respond in under 1 second after the initial startup delay.<\/p>\n\n\n\n
The
\/health<\/code> endpoint lets you verify the server is running without triggering an inference call useful for monitoring and load balancer health checks.<\/p>\n\n\n\nStep 11 – Open the Firewall Port<\/h3>\n\n\n\n
Allow traffic on port 5000:<\/p>\n\n\n\n
ufw allow 5000\nufw reload<\/code><\/pre>\n\n\n\nIf UFW is not active, enable it first:<\/p>\n\n\n\n
ufw allow ssh\nufw allow 5000\nufw enable<\/code><\/pre>\n\n\n\nAlways allow SSH before enabling UFW or you lock yourself out of the server.<\/p>\n\n\n\n
Step 12 – Start the API Server<\/h3>\n\n\n\n
python3 api.py<\/code><\/pre>\n\n\n\nThe server starts and shows:<\/p>\n\n\n\n
Loading model...\nModel loaded. API ready.\n * Running on http:\/\/0.0.0.0:5000<\/code><\/pre>\n\n\n\nThe model load takes 10-30 seconds depending on server speed. Once you see “API ready”, the endpoint accepts requests.<\/p>\n\n\n\n
Step 13 – Test the Endpoint<\/h3>\n\n\n\n
Open a second terminal session on your local machine and run:<\/p>\n\n\n\n
curl -X POST http:\/\/127.0.0.1:5000\/generate -H \"Content-Type: application\/json\" -d '{\"prompt\":\"The future of AI is\"}'<\/code><\/pre>\n\n\n\nResponse:<\/p>\n\n\n\n
{\n \"prompt\": \"The future of AI is\",\n \"response\": \"The future of AI is deeply tied to how organizations structure their data pipelines and invest in the infrastructure that supports real-time decision making...\",\n \"model\": \"gpt2\"\n}<\/code><\/pre>\n\n\n\nTest the health endpoint:<\/p>\n\n\n\n
curl http:\/\/YOUR_SERVER_IP:5000\/health<\/code><\/pre>\n\n\n\nResponse:<\/p>\n\n\n\n
{\"status\": \"ok\", \"model\": \"gpt2\"}<\/code><\/pre>\n\n\n\nYour AI inference API is live and responding to requests from any machine that can reach your server’s IP.<\/p>\n\n\n\n
![\"model](\"https:\/\/www.ucartz.com\/blog\/wp-content\/uploads\/2026\/05\/model-output-1024x143.webp\")
<\/figure>\n\n\n\nStep 14 – Keep the Server Running After Logout<\/h3>\n\n\n\n
When you close your terminal, the Python process stops. Use
screen<\/code> ornohup<\/code> to keep it running:<\/p>\n\n\n\napt install screen -y\nscreen -S ai-api\npython3 api.py<\/code><\/pre>\n\n\n\nDetach from the screen session with CTRL+A then D. The API keeps running after you close your terminal.<\/p>\n\n\n\n
Reattach later with:<\/p>\n\n\n\n
screen -r ai-api<\/code><\/pre>\n\n\n\nFor a permanent production setup, create a systemd service:<\/p>\n\n\n\n
nano \/etc\/systemd\/system\/ai-api.service<\/code><\/pre>\n\n\n\n[Unit]\nDescription=Hugging Face AI API\nAfter=network.target\n\n[Service]\nUser=root\nWorkingDirectory=\/root\/ai-model\nEnvironment=PATH=\/root\/ai-model\/venv\/bin\nExecStart=\/root\/ai-model\/venv\/bin\/python3 api.py\nRestart=always\nRestartSec=10\n\n[Install]\nWantedBy=multi-user.target<\/code><\/pre>\n\n\n\nEnable and start:<\/p>\n\n\n\n
systemctl daemon-reload\nsystemctl enable ai-api\nsystemctl start ai-api\nsystemctl status ai-api<\/code><\/pre>\n\n\n\nThe API now starts automatically on every server boot and restarts automatically if it crashes.<\/p>\n\n\n\n
Switch to Better Models<\/h3>\n\n\n\n
GPT-2 is good for learning but limited in quality. Once your setup is confirmed working, switch to better models by changing one line in your script.<\/p>\n\n\n\n
For better text generation:<\/p>\n\n\n\n
generator = pipeline(\"text-generation\", model=\"distilgpt2\") # smaller, faster\ngenerator = pipeline(\"text-generation\", model=\"EleutherAI\/gpt-neo-125m\") # better quality<\/code><\/pre>\n\n\n\nFor text summarization:<\/p>\n\n\n\n
summarizer = pipeline(\"summarization\", model=\"facebook\/bart-large-cnn\")<\/code><\/pre>\n\n\n\nFor question answering:<\/p>\n\n\n\n
qa = pipeline(\"question-answering\", model=\"deepset\/roberta-base-squad2\")\nresult = qa(question=\"What is Hugging Face?\", context=\"Hugging Face is an AI company...\")<\/code><\/pre>\n\n\n\nFor text classification:<\/p>\n\n\n\n
classifier = pipeline(\"sentiment-analysis\", model=\"distilbert-base-uncased-finetuned-sst-2-english\")<\/code><\/pre>\n\n\n\nEach model has different RAM requirements. Check the model page on huggingface.co for the model size before downloading it on a memory-constrained VPS.<\/p>\n\n\n\n
Common Mistakes That Break This Setup<\/h3>\n\n\n\n
Starting with large models on a small VPS<\/strong> – A 7B parameter model needs 14GB of RAM minimum on CPU. If you load a model that exceeds available RAM, the process gets killed silently or the server becomes unresponsive. Start with GPT-2 or DistilGPT-2, confirm everything works, then scale up the VPS and model together.<\/p>\n\n\n\n
Skipping the virtual environment<\/strong> – Installing Transformers and PyTorch system-wide conflicts with Ubuntu’s system Python packages. Activate the virtual environment every time before running scripts. If you see import errors on a model you know is installed, the virtual environment is probably not activated.<\/p>\n\n\n\n
Forgetting to open port 5000<\/strong> – The Flask server starts and shows “Running on 0.0.0.0:5000” but external requests time out. Check UFW and your VPS provider’s firewall panel. Both need port 5000 open for external access.<\/p>\n\n\n\n