Mohit Garg

Why I built Imajev, an open model for the decisions people still make by hand

I’ve spent years drawing process maps for refunds, returns and customer support, including for Fortune 500 companies. Every one of those maps had the same shape. Boxes that software already handled, joined by diamonds. The diamonds are the decision points: does this refund meet the policy, is this return the item we shipped, which team owns this ticket.

And on every map, the diamonds were handled by a person.

They stay manual because putting ambiguity into code is very hard. A rule engine needs every case written down in advance, and real cases never read like the rule book. So a person looks at the ticket, the photo or the record and makes the same call for the hundredth time that day. It’s slow, the answer changes with who’s on shift, and when they get one wrong, the customer finds out first.

Then Jev came out

Jev, from TypeSafe, is a model built for exactly this kind of question. You give it the facts and a closed question with the answers you’ll accept, and it gives you a probability for each answer. Not an essay. A number your system can put in an if.

When I saw it, I could see it sitting on those diamonds. But Jev only reads text, and many of these decisions start with a photo: the item that came back, the part on the line, the picture a seller uploaded. It’s also a hosted service, so the photo and the customer record leave your network for every call.

So I set out to build the same idea for text and images, in one open model you run on your own servers. That became Imajev.

What it does

You send the evidence (a photo, a record, some text) and the questions you care about. For each question you list the answers you’ll accept. Imajev returns a probability for every answer, plus one for can’t tell.

A listing says the shoes are red; the photo shows beige suede boat shoes. Imajev names listing.color as the contradicted field at 0.950, unknown at 0.007, and the app holds the listing.

Here a marketplace listing says the shoes are red. The photo shows beige suede. Imajev names the colour field as the one the photo contradicts, at 0.95, and the app holds the listing instead of publishing it.

The can’t tell part is the one I care about most. Ask it to find a white or beige bag in a closet that only has a red and a black backpack, and it puts 0.89 on “unknown” instead of guessing. That’s what a careful person would say. A model with no trained way to say can’t tell has to pick something.

It got worse before it got better

My first big fine-tune went badly. I trained the 9B model on about 500,000 short, human-labelled decisions, and it got worse at reasoning. On the hard questions in JevBench, the benchmark for Jev-style models, it fell from 64.9% to 42.3%. More data had made it dumber.

I spent the next couple of weeks generating hard questions with open models and keeping a question only when two different models agreed on the answer. 9,368 of 13,386 survived that filter. Training on them brought the 9B back to 67.6%.

The lesson I took: volume of easy examples doesn’t teach judgement. Hard examples you can trust do.

The whole project, every training run included, cost about $676 of rented GPU time. I wrote the code with Claude as a co-author.

The part a business should care about

A benchmark score doesn’t tell an operations lead what to do on Monday. A threshold does.

You decide how sure the model must be before your system acts on its own. Everything below that line goes to a person, including every can’t tell.

At a 90% threshold, imajev-4b decides 58% of the 279 ImajevBench questions automatically, gets 97.5% of those right, and sends 42% to a person.

At 90%, the 4B model handles 58% of the benchmark’s questions on its own and gets 97.5% of those right. The other 42% go to a person. Raise the bar to 99% and it handles 40%, with every one of those right. You pick the trade-off from what a mistake costs you, not from what the model would like to do.

That’s the shape I wanted from the start: automate what’s clear, and route the rest to the people who were doing all of it before.

What the independent boards say

Each of these leaderboards is run by its own maintainer, not by me.

Imajev rankings: #1 of 91 on JevBench v1.4.2.2, ahead of Jev 1.13.0; #1 of 49 on Image JevBench v0.1.3, ahead of Jev-Omni (12B); #3 of 56 on DecisionBench (eng, v1), ahead of GPT-5.6 Luna.

  • #1 of 91 on JevBench v1.4.2.2 (scored 27 Sep 2026): 67.37, against 63.29 for Jev 1.13.0 itself.
  • #1 of 49 on Image JevBench v0.1.3 (28 Sep 2026), ahead of Jev-Omni, a 12B model.
  • #3 of 56 on DecisionBench, ahead of GPT-5.6 Luna and DeepSeek V4.1 Flash. The two above it are the benchmark team’s own models.
  • Zero invalid answers: 23,900 of 23,900 on DecisionBench. Imajev reads its answer off the options you gave it in one pass instead of writing text, so it can’t return a malformed answer.

On JevBench’s own estimate, it costs about $0.022 per 1,000 decisions to run.

Where it falls short

I’d rather you hear this from me:

  • It doesn’t reason step by step. On the hardest text questions, reasoning models still beat it.
  • It’s English only, and takes at most two photos per request.
  • Comparing two photos is its weakest visual task.
  • Can’t tell is trained, not guaranteed. In one test, a blank payment note was read as “not paid” instead of “unknown”.

The benchmark I built for it, ImajevBench, uses AI-generated images and hasn’t had a human audit yet. So treat every number above as a starting point.

If you run a process like this

Pick one decision your team makes hundreds of times a week. Pull a few hundred real cases with the right answer, including the ones where the honest answer is “we couldn’t tell”. Run them through the model, and choose your threshold from what a wrong call costs you. That afternoon tells you more than any leaderboard.

Imajev is open source under Apache-2.0: the code is on GitHub, the weights are on Hugging Face, and there’s a live demo where you can swap a photo and watch the decision change.

And if you’d rather someone else wired it into your queue, that’s the work I do.