Build your own feedback on the model's output
Quran Lab ships no scorer. What the heads are, the thresholds that ship as data, and what your app builds itself.
The SDK is an inference engine: audio in, the model's output out. It never compares a recitation with an expected ayah and never says that something was wrong. Quran Lab ships no scorer. What a learner sees as feedback is your app's work, built on what the model heard. This page says what you get and what is yours to build.
What the SDK gives you
For every take:
- The decoded layer. What the model heard, ready to show:
text,tokens(each withid,name,firstMs,peakMs,lastMsandconfidence),words, and per 40 ms frame the top token with its runner-up and the margin between them.settledis the count of leading tokens that will not change any more in this take;isFinalis true on the last one. - The heads. The raw outputs of the network, per chunk while the take runs and for the whole take when it ends.
- The vocabulary.
model.tokens: 38 tokens. Blank, the word boundary, the 28 consonants, the ikhfa nasal, the short vowels a, i and u, the long vowels aa, ii and uu, and one token for foreign sounds.
And two files of read-only data that come with the model. The SDK never applies or reads either one:
- The calibration (
model.calibration): the thresholds Quran Lab measured for this model. - The reference data (
model.reference): the expected units of every ayah.
Where the expected units and the rules come from: model.reference is made from Quran Lab's reference for the phonetics and tajweed rulings of the Quran, which also gives each unit its phones, its rules and where the classical books state them. The reference shows how to join it to the model's units, with a worked example.
The heads
Twelve outputs, with the same names on every platform. A session computes log_probs, inter1_log_probs, timing_log_probs, gem, attr_qalqalah and conf unless you ask for others.
| Head | Rate | What it is |
|---|---|---|
log_probs |
40 ms | The final CTC over the 38 tokens: what was said |
inter1_log_probs |
40 ms | CTC from an early layer. It stays closer to the sound |
inter4_log_probs |
40 ms | CTC from a middle layer, for diagnosis |
id_log_probs |
20 ms | Identity CTC over the same tokens, at finer time resolution |
timing_log_probs |
20 ms | When each sound is held: durations, and how long a lengthening lasts |
gem |
20 ms | Gemination: single or held (shadda) |
attr_tafkheem |
20 ms | Thin, heavy or low heavy |
attr_ghunna |
20 ms | None, plain, geminate, ikhfa, idgham with ghunna |
attr_qalqalah |
20 ms | None, sughra, kubra |
attr_face |
20 ms | None, imala, tasheel, sakt |
boundary |
20 ms | A logit per frame for where a unit begins |
conf |
20 ms | A confidence logit per frame |
The CTC heads are log-probabilities over the tokens; gem and the attr_ heads are probabilities per class; boundary and conf are logits (apply a sigmoid).
The model uses no language model anywhere: no transducer, no rescoring, no text prior. A language model quietly corrects what a reciter said, which is exactly what you do not want here.
The thresholds ship as data
calibration.json travels with the model, signed and verified like every other file, and the SDK hands it to you exactly as shipped. It belongs to one model and one vocabulary; model_id and tokens_sha256 in it say which. It holds:
- the
classesof single-unit difference a reading can test:vowel_sub,vowel_del,vowel_ins,short_to_long,long_to_short,cons_sub,cons_delandinsertion; - four named sets of thresholds (
strict,balanced,sensitive,very_sensitive), each fitted to a target false-alarm rate on correct recitation; - per context,
fusion: the two heads the thresholds apply to (log_probsandinter1_log_probs), the weightaper class, howinter1_log_probsis prepared (the word boundary merged into blank) and the clip of the values; - per context and class, a threshold in nats for each named set, and a
status:validated,experimental,unverifiedoroff; - per context, the numbers for lengthening (
madd, read fromtiming_log_probs),qalqalah(fromattr_qalqalah),gemination(fromgem) and, when the build has it, a per-word readout (word).
unverified means that the thresholds were fitted on development data and have not been read on held-out data yet, so nothing is claimed about their false-alarm rate. A build on an early channel can carry it on a class or on the madd, qalqalah and gemination numbers. Treat it as you treat experimental: usable, not promised. It never means that a held-out read passed.
A build may also come without the per-word readout: its calibration then has no word block. Check that the block is there before you read it.
The thresholds may differ from one model version to the next. Read them from the loaded model; never copy them into your code.
What your app builds itself
- The expected units of the ayah.
model.referenceis a lookup: surah and ayah in, the unit ids and the word boundaries out. It is not part of the first download; fetch it once. Its ids are those ofmodel.tokens. - The evidence at each place. For each position of the expected sequence and each single-unit alternative of a class, compute the full-sequence CTC forward log-probability of the expected sequence and of the alternative, on
log_probsand again oninter1_log_probsprepared as the calibration says. The log-likelihood ratio of a head is log P(best alternative) minus log P(expected). Fuse the two heads with the class's weight,a * llr(log_probs) + (1 - a) * llr(inter1_log_probs), clip it, and compare it with the calibration's threshold for that context and class. - Lengthening, qalqalah and gemination. A lengthening is counted from
timing_log_probs: the span a long vowel is held against the median short-vowel span of the take, with the cuts in the calibration. Qalqalah is read fromattr_qalqalahover the frames after the letter, and gemination fromgemover the letter's span, each against its value in the calibration. - Which set of thresholds a learner gets, and any change per class. That is your app's setting.
- The words a learner reads. How a difference is shown, and when, is yours.
Each SDK's docs/migrating-to-3.md describes the alignment and the ratios step by step.
Limits you should design around
From the model card, for TajweedFormer V4:
- The calibration of this release marks gemination
offand qalqalahexperimental; another build may mark them otherwise, so read each status from the loaded model. Theattr_ghunnaandattr_tafkheemheads have no validated thresholds in this release: treat them as raw signals. - Lengthening of the lazim kind has too few calibration sites to be reliable, and muttasil is the weakest lengthening class.
- Phone recordings and children's voices are the weakest case for every threshold.
- The thresholds were fitted for the fused value of two heads. One head alone, or the greedy text, is not evidence that a learner recited something wrong.
What the licence asks of your screen
Never present the model's output as a religious ruling on anyone's recitation, and never charge for the model, for access to it or for a feature it powers. Your own teaching and work stay yours.
Quran Lab also encourages, but does not require, one line where your app shows feedback built on the model, for example: "Automatic tajweed feedback can be wrong. It does not replace a qualified teacher." Any honest wording is fine, and whether to show it is up to you.
Analytics for what you build
The SDK counts what it measures itself: takes, seconds of audio, speech and silence, latency, tokens. What it cannot know without your feedback, your app may report; nothing is sent if it does not:
session.reportRecited(1, 1, 7); // surah, first ayah, last ayah of this take
client.analytics.reportChecks('vowel_sub', 31, 1); // your rule, times checked, times flaggedBoth go into the day's anonymous totals and are marked on your app's page as reported by the app.
Evaluate the model on your own data
To measure how the model does on your own recordings, for a benchmark, an accuracy test or a comparison with another model, run them through the SDK. It gives exactly the model's outputs, so an evaluation needs no raw weights, and a raw-weights request made to evaluate the model is answered with this section instead of being reviewed.
On a desktop, with the C SDK. tajweedformer-eval comes with the C SDK from 3.1.4 and in 4.0. It is a Windows x64 download on the release page and builds from source on Linux; there is no macOS build. It needs a desktop key and the identity the key was made for (--app-id); the key is --key or QURANLAB_KEY. Give it a folder of audio files, a list of files or one file, and your references; once the model is installed it runs offline:
tajweedformer-eval --app-id <app id> clips/ --refs refs.csv --out results.jsonl- Input: WAV (8 to 32-bit PCM, or float 32 or 64) or FLAC at any sample rate. A file over 20 minutes is evaluated on its first 20 minutes and marked so.
- References: CSV
file,referenceor JSONL{"file", "reference"}. A reference is an ayah range (112:1,2:255-257,2:255-3:2) or the expected units. - Output: one line of JSON per file, with the units heard, their timings and confidence, a summary of each head, the processing time and the real-time factor, and with a reference the unit error rate (
ref,hyp,sub,del,ins,rate) and the word error rate (ref,wrong,rate); then one line with the totals. Other flags:--context,--threads,--heads,--storage,--version. - Exit code 0 when every file was evaluated, 1 when some failed, 2 when nothing ran.
The model gives no word boundaries, so the word error rate is counted over your reference's words: the units are aligned, and a reference word is wrong when any of its units is substituted or deleted, or an inserted unit follows it.
In your project, with the platform's SDK. On Android, Apple (also on a Mac) or the web, write a test that plays each clip through the SDK (model.session() and session.push(samples)), keeps the final chunk.decoded of each clip and compares it with your reference.
Compare in the model's own units (model.tokens); for a recited ayah, model.reference gives the expected units. Count the substitutions, deletions and insertions against the reference, and give their sum over the length of the reference as the error rate, per clip and in total. A development key keeps these runs out of your app's figures.