How to Fine-Tune a Custom AI Model in Vincony: A Step-by-Step Walkthrough

# How to Fine-Tune a Custom AI Model in Vincony: A Step-by-Step Walkthrough

If you've read our conceptual fine-tuning guide and decided fine-tuning is the right tool for your use case, this post is what comes next. Skip the theory — here you'll learn exactly how to create and run a fine-tuning job inside Vincony's Fine-Tuning Studio, from preparing your first dataset to chatting with the finished model.

---

How Fine-Tuning Works in Vincony

The Fine-Tuning Studio is a three-step wizard. You give a job a name, choose a base model, upload training data, review costs, then click Create Job. Vincony handles the rest — queuing, training, and surfacing the result as a model ID you can immediately test in Chat or route through your BYOK setup.

Jobs have four statuses: Pending (queued, not yet started), Running (training in progress), Completed (model ready), and Failed (something went wrong — retry is one click away). The dashboard auto-polls and shows a live progress bar with an estimated time remaining while a job runs.

Fine-tuning is available on Starter plans and above. The credit cost is the base model cost plus one credit per ten training rows — see pricing for the full breakdown. Free plan users can explore the UI but need to upgrade to submit jobs.

---

Step 1 — Prepare Your Training Data

The wizard accepts data in four ways. Understanding each option upfront saves backtracking.

Option A: Upload a JSONL file

Drag and drop a .jsonl file onto the upload zone. Each line must be a valid JSON object with a messages key containing an array of conversation turns. Each turn needs a role (one of system, user, or assistant) and a content string. Every line in the file is one independent training example in that shape.

The UI validates your file on drop and immediately shows a Validation Summary: how many lines parsed, how many failed, and what the errors are (missing keys, malformed JSON, wrong roles). A Training Data Preview shows the first several rows inline. A Data Quality Score badge grades your dataset on diversity and completeness so you can catch thin or repetitive datasets before paying to train on them.

Aim for at least 50 examples before submitting. 200–500 is a solid starting point for most style or tone jobs.

Option B: Use AI to generate examples for you

If you don't have training data yet, click Generate with AI (the Sparkles button) at the top of the data step. A dialog opens where you:

• Describe your use case in plain language — for example, "a customer support agent for a SaaS billing product that responds in a friendly but brief tone."
• Set how many examples to generate (the default is 10; you can go higher).
• Click Generate.

Vincony calls your AI gateway and returns a table of system / user / assistant triples. You can edit any row inline, delete poor examples, and add your own rows manually before clicking Use These Examples. The generated rows are converted to JSONL automatically and dropped straight into the wizard — no file exports needed.

This is the fastest path from zero to a first training run — generate a batch, curate it, then supplement with real production data in later iterations.

Option C: Start from a template

Three built-in templates appear below the upload zone:

• Customer Support Tone — empathetic, professional responses to common support scenarios
• Technical Writing — structured documentation and explanations
• Brand Voice — casual, witty, and confident copy

Click any template to load it instantly. The Validation Summary and Quality Score update to reflect the template data, and you can preview all rows before proceeding. Templates are a fast way to verify the workflow end-to-end with known-good data before investing time in a custom dataset.

Option D: Paste a URL

If your JSONL file is hosted somewhere accessible (an S3 bucket, a public GitHub raw URL, an internal API endpoint), paste the URL into the Or paste a URL field. The wizard will reference that URL when it submits the job.

---

Step 2 — Create the Job (the Three-Step Wizard)

Click New Fine-Tune Job (or press N from anywhere on the page) to open the wizard.

Wizard Step 1: Details

• Job Name — give it something descriptive, like "Support Tone v1" or "Legal Summary Formatter." You can rename jobs inline from the dashboard later, so don't overthink this.
• Base Model — choose from three options:

Start with GPT-5 Mini for most first runs. You can always clone the job and retry with a larger model if the results disappoint.

Click Next once you've filled in the name.

Wizard Step 2: Training Data

This is where you supply your dataset using any of the four options described above. Once data is loaded, the Validation Summary, Quality Score, and row preview all appear inline inside the wizard. Fix any validation errors before proceeding — the wizard won't block you, but invalid rows are silently skipped during training.

Click Next when your data looks good.

Wizard Step 3: Review and Submit

The review screen shows a summary card:

• Job Name and Base Model confirm what you entered
• Training Data displays the file name or URL
• Training Rows shows how many valid examples were detected
• Estimated Time is shown as 15–45 minutes (actual time scales with dataset size and model)
• Estimated Cost in credits, broken down as base model cost plus the per-row data cost

Two optional toggles appear below the summary:

• Schedule for later — flip this on and pick a date/time to queue the job for a future run rather than starting immediately. Useful if you want training to happen off-peak or you're still refining data.
• Notify on completion — when enabled, Vincony shows a toast notification when the job finishes (or fails) even if you've navigated away from the page.

When you're satisfied, click Create Job (or Schedule Job if scheduling is on). The dialog closes and your new job appears at the top of the dashboard.

---

Step 3 — Monitor Training Progress

Once a job is running, its card shows a Training Timeline — a segmented progress bar that fills as training advances. Below it, the percentage complete and an estimated time remaining update in real time. You don't need to stay on the page; the notification toggle will alert you when it's done.

If a job fails, an error message appears in red on the card. A Quick Retry button (the circular arrow icon) clones the failed job and re-submits it immediately — handy for transient infrastructure failures without having to re-enter all your settings manually.

You can also use the Clone button (copy icon) on any job to duplicate its configuration and open the wizard pre-filled. This is the recommended workflow for iterating: clone, adjust the dataset or switch the base model, and submit a new run.

---

Step 4 — Use Your Finished Model

When a job reaches Completed status, a result_model_id badge appears on the card. Three action buttons become available:

Test in Chat

Click Test in Chat to jump directly to /os/chat with your fine-tuned model pre-selected. This is the fastest way to do a smoke test — ask it a few representative questions and compare the responses to the base model.

Compare

Click Compare to expand a Comparison Playground directly below the job card. Type a prompt once and see the fine-tuned model's response alongside the base model's response in a side-by-side view. This is the most efficient way to validate that training actually changed the behavior in the direction you intended.

Auto-Evaluate

Click Auto-Evaluate to open the automated evaluation dialog. You provide a set of test prompts and expected outputs, and Vincony runs them through both the base model and your fine-tuned model, scoring each response. The result is a structured report you can use to decide whether the model is ready for production or needs another training iteration.

---

Managing Your Jobs Dashboard

Once you have more than a handful of jobs, a few dashboard features become useful:

• Search (press / to focus the search bar) filters jobs by name, base model, or provider in real time.
• Status filter lets you show only Pending, Running, Completed, or Failed jobs.
• Sort by newest, oldest, or name alphabetically.
• Group by status or base model to visually cluster related jobs.
• Bulk select — check the checkbox on any job card to select it, then use the bulk action bar to delete multiple jobs at once.
• Export config (the download icon) saves a job's full configuration as a JSON file — useful for version-controlling your fine-tuning runs or sharing configs with teammates.
• Share config (the share icon) copies the job configuration to your clipboard as formatted JSON.
• Keyboard shortcuts — press ? at any time to see the full shortcuts reference overlay.

---

Tips and Troubleshooting

Validation errors on upload. The most common causes are missing the messages key, using a role other than system, user, or assistant, or having malformed JSON (trailing commas, unescaped quotes). Fix these in your source file and re-drop it — the validator re-runs instantly.

Low Data Quality Score. This usually means too few examples, or examples that are too similar to each other. Add more variety: different phrasings of the same question, edge cases, and failure modes you want the model to handle gracefully.

Job fails immediately. Check the error message on the card. If it's a data error, use the Edit Data button (available on pending jobs) to fix the training file in place without re-creating the job. If it's an infrastructure error, Quick Retry handles it.

Results look similar to the base model. Your dataset may not have enough signal, or the behavior you want is already present in the base model (in which case a well-crafted system prompt via BYOK may be sufficient). Try increasing the dataset to 500+ examples with more pronounced, consistent examples of the target behavior.

Credits running low. GPT-5 Nano at 5 base credits is the most economical option for experimentation. Check the live credit estimate on the Review step before submitting — the data-row cost only adds up on very large datasets (100+ rows adds 10+ credits on top of the base cost).

---

Ready to Build Your First Model?

Head to Fine-Tuning Studio and click New Fine-Tune Job. If you're starting from scratch with no dataset, the AI Data Builder gets you to a first job in under five minutes. Starter plan and above — check pricing if you need to upgrade first.

For a deeper understanding of when fine-tuning is the right tool versus RAG or prompt engineering, see the companion Fine-Tuning Guide. For using your finished model with your own API key and infrastructure, see BYOK.