Leiden University Centre for Linguistics (LUCL), Leiden University, The Netherlands · ICASSP 2027 submission
A neural encoder estimates frame-level controls; a deterministic source–filter synthesizer converts them back into speech. Every control is a synthesizer coefficient (F0, Rd, one resonator per formant), so each cue is edited directly, with a model trained on at most 1 h of one speaker.
★ editable · dashed: learned, not exposed
Error of the edited cue when F0–F3 are scaled by 0.7–1.3 (60 sentences per corpus).
Unedited resynthesis by each system. All files at −26 dBFS.
| Item | Natural | ARIS | WORLD | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|---|---|
| 大量土方涌进犬舍。dà liàng tǔ fāng yǒng jìn quǎn shè000255 | reference | LSD 6.66 | LSD 6.86 | LSD 46.48 | LSD 9.47 |
| 欢迎各位朋友常来逛逛聊聊。huān yíng gè wèi péng you cháng lái guàng guang liáo liao000195 | reference | LSD 7.10 | LSD 7.29 | LSD 47.87 | LSD 8.95 |
| 主要有鲤鱼、银鲫、鲶鱼、草鱼、泥鳅等。zhǔ yào yóu lǐ yú yín qí nián yú cǎo yú ní qiu děng000930 | reference | LSD 7.15 | LSD 7.33 | LSD 47.38 | LSD 9.08 |
LSD in dB (lower = closer to the recording)
| Item | Natural | ARIS | WORLD | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|---|---|
| You understand what I mean, Mister Verloc?secretagent_02_conrad_0699 | reference | LSD 6.93 | LSD 7.56 | LSD 33.05 | LSD 8.81 |
| What with one sort of attachment and another you are doing away with your usefulness.secretagent_02_conrad_0893 | reference | LSD 6.94 | LSD 7.54 | LSD 36.47 | LSD 8.93 |
| It isn't very wise to call me up like this.secretagent_02_conrad_0616 | reference | LSD 6.98 | LSD 7.69 | LSD 36.82 | LSD 8.72 |
LSD in dB (lower = closer to the recording)
| Item | Natural | ARIS | WORLD | KlattGrid |
|---|---|---|---|---|
| pǐF024_pi3 | reference | LSD 6.71 | LSD 7.57 | LSD 33.13 |
| mǎiF024_mai3 | reference | LSD 6.75 | LSD 7.58 | LSD 34.09 |
| miǎoF024_miao3 | reference | LSD 6.80 | LSD 7.68 | LSD 34.73 |
LSD in dB (lower = closer to the recording)
| Item | Natural | ARIS | WORLD | KlattGrid |
|---|---|---|---|---|
| planetaryPLANETARY | reference | LSD 7.04 | LSD 7.10 | LSD 36.36 |
| vileVILE | reference | LSD 7.10 | LSD 7.01 | LSD 32.42 |
| unnaturalUNNATURAL | reference | LSD 7.11 | LSD 7.44 | LSD 36.53 |
LSD in dB (lower = closer to the recording)
| Item | Natural | ARIS | WORLD | KlattGrid |
|---|---|---|---|---|
| beitsbeits | reference | LSD 7.35 | LSD 7.59 | LSD 36.33 |
| gekeurdgekeurd | reference | LSD 7.41 | LSD 7.71 | LSD 40.07 |
| tijtij | reference | LSD 7.42 | LSD 7.57 | LSD 36.36 |
LSD in dB (lower = closer to the recording)
The same sentence: natural, ARIS and HiFi-Glot (FT).
“大量土方涌进犬舍。”
“You understand what I mean, Mister Verloc?”
Unedited resynthesis of 15 random sentences per corpus (seed 0). All files at −26 dBFS.
| Item | Natural | ARIS | HiFi-Glot (FT) |
|---|---|---|---|
| 仿针织、粗花呢等印花面料的巧妙运用。fǎng zhēn zhī cū huā ní děng yìn huā miàn liào de qiǎo miào yùn yòng000055 | |||
| 莽莽乡野，万水千山，铿锵承诺。máng mǎng xiāng yě wàn shuǐ qiān shān kēng qiāng chéng nuò000343 | |||
| 这于商丘又有何益呢？zhè yú shāng qiū yòu yǒu hé yì ne000400 | |||
| 普陀山既以海天壮阔取胜，又以山林深邃见长。pǔ tuó shān jì yí hǎi tiān zhuàng kuò qǔ shèng yòu yǐ shān lín shēn suì jiàn cháng000469 | |||
| 陈奕迅透露太太徐濠萦因忙于跑步，故未能出席捧场。chén yì xùn tòu lu tài tai xú háo yíng yīn máng yú pǎo bù gù wèi néng chū xí péng chǎng000509 | |||
| 血从她头顶往外冒个不停。xuè cóng tā tóu dǐng wǎng wài mào ge bù tíng000538 | |||
| 羊、狗年平平，总算形象讨喜。yáng gǒu nián píng píng zǒng suàn xíng xiàng táo xǐ000559 | |||
| 届时王晶又能笑得合不拢嘴了。jiè shí wáng jīng yòu néng xiào de hé bu lóng zuǐ le000636 | |||
| 少数患者出现兴奋，或有片断荒谬的妄想与幻觉。shǎo shù huàn zhě chū xiàn xīng fèn huò yǒu piàn duàn huāng miù de wàng xiáng yǔ huàn jué000647 | |||
| 秋保的脚部等处也负有轻伤。qiū bǎo de jiǎo bù děng chù yě fù yǒu qīng shāng000679 | |||
| 肚子饿得咕噜咕噜叫。dù zi è de gū lu gū lu jiào000779 | |||
| 恶意骚扰使得大量急救资源被占用。è yì sāo rǎo shǐ dé dà liàng jí jiù zī yuán bèi zhàn yòng001013 | |||
| 会上，与会者畅所欲言，发言活跃。huì shàng yù huì zhě chàng suǒ yù yán fā yán huó yuè001060 | |||
| 他嘴巴里说着脏话，紧攥拳头朝我挥舞。tā zuǐ ba lǐ shuō zhe zāng huà jǐn zuàn quán tou cháo wǒ huī wǔ001121 | |||
| 别人上山靠脚走，他却要翻着跟头上山去。bié rén shàng shān kào jiáo zǒu tā què yào fān zhe gēn tou shàng shān qù001143 |
| Item | Natural | ARIS | HiFi-Glot (FT) |
|---|---|---|---|
| He found at home the ease of his body and the peace of his conscience,secretagent_01_conrad_0100 | |||
| and his general get-up was that of a well-to-do mechanic in business for himself.secretagent_02_conrad_0066 | |||
| an employer of labour in a small way.secretagent_02_conrad_0068 | |||
| who was going out with the papers in his hand:secretagent_02_conrad_0272 | |||
| I daresay you are intelligent enough for your work.secretagent_02_conrad_0404 | |||
| And at the same time he had the advantage of seeing his own face,secretagent_02_conrad_0471 | |||
| and the First Secretary of the Embassy raised his head to look at him.secretagent_02_conrad_0555 | |||
| the exasperation of a hungry man, an act of social revenge.secretagent_02_conrad_0733 | |||
| it would be really telling if one could throw a bomb into pure mathematics.secretagent_02_conrad_0798 | |||
| exclaimed the other in unfeigned astonishment.secretagent_02_conrad_0874 | |||
| and sat down on a wooden chair that stood there.secretagent_02_conrad_0934 | |||
| The old terrorist turned slowly his head on his skinny neck from side to side.secretagent_03_conrad_0062 | |||
| optimism- His laborious wheezing stopped,secretagent_03_conrad_0089 | |||
| the economic condition of the world responsible for the past and shaping the future;secretagent_03_conrad_0147 | |||
| the symbolism of a mad art attempting the inconceivable.secretagent_03_conrad_0167 |
One cue scaled at a time; each button shows the measured change.
Button = measured change of the edited cue: ±2 pts 2–5 pts > 5 pts from the requested ±20%.
xiě hòu fàng zài zuò wèi de yòu biān yòng yǐ zì jǐng
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
fēi yào shuō shì bàn yǎn hè lóng de dù yuán ne
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
| Scale | ARIS | KlattGrid | HiFi-Glot (FT) |
|---|---|---|---|
| ×0.8 | |||
| ×1 unedited | |||
| ×1.2 |
One natural token, one control moved in equal steps. Values under each step are measured.
ǎ → yǐ 啊 (interjection) → 椅 ‘chair’
F1 and F2 move from the natural ǎ targets towards those of yǐ from the same speaker, with targets equally spaced in Bark.
F3 is also interpolated in Bark to avoid F2–F3 overlap near yǐ; F0 and source controls are held fixed. These are formant edits, not a complete reproduction of the natural endpoint vowel.
Source of the Bark conversion used for target spacing; not a validation of this Mandarin vowel path. Traunmüller, H. (1990). Analytical expressions for the tonotopic sensory scale. JASA, 88(1), 97–100.
ǎ → yǐ
ǎ → wǔ 啊 (interjection) → 五 ‘five’
F1 and F2 move from the natural ǎ targets towards those of wǔ from the same speaker, with targets equally spaced in Bark.
F3, F0 and source controls are held fixed, so the endpoint is an F1–F2 approximation rather than a full reproduction of wǔ. F2 cannot go below 700 Hz, the lower end of its range, so steps 7–9 have nearly the same F2.
Source of the Bark conversion used for target spacing; not a validation of this Mandarin vowel path. Traunmüller, H. (1990). Analytical expressions for the tonotopic sensory scale. JASA, 88(1), 97–100.
ǎ → wǔ
lemon [ˈlɛmən] ‘lemon’
F1, F2 and F3 of ‘lemon’ are multiplied by a shared factor from 0.85 to 1.15.
F0 and source controls stay fixed, and the eight residual pole pairs are not scaled. This is a partial spectral manipulation, not a complete vocal-tract length transformation; a change in perceived speaker size is not established by this demo.
Background on spectral scaling and perceived speaker size. This demo scales only F1–F3 and does not reproduce that experiment. Smith, D. R. R., Patterson, R. D., Turner, R., Kawahara, H., & Irino, T. (2005). The processing and perception of size information in speech sounds. JASA, 117(1), 305–318.
×0.85 · lower formants → ×1.15 · higher formants
F0 contours interpolated in semitones; spectral envelope, timing and voicing unchanged.
dá → dǎ ‘answer’ (答 dá) vs ‘hit’ (打 dǎ)
F0 is interpolated in semitones between natural dá and dǎ contours from the same speaker.
The contours are time-normalised to the base token's voiced span, with period-doubled frames in the Tone 3 dip bridged. The base token's spectral-envelope, voicing, duration and amplitude controls are held fixed; this illustrates an F0 edit, without a claim about which tone pair is most confusable.
dá → dǎ
yī → yí ‘clothing’ (衣 yī) vs ‘aunt’ (姨 yí)
F0 is interpolated in semitones from the natural yī contour towards the natural yí contour of the same speaker.
The target contour is time-normalised to the base voiced span; spectral-envelope, duration and amplitude controls are held fixed.
Background on level–rising pitch perception and language experience. The study used linear pitch ramps; this demo interpolates natural contours and does not establish categorical perception. Xu, Y., Gandour, J. T., & Francis, A. L. (2006). Effects of language experience and stimulus complexity on the categorical perception of pitch direction. JASA, 120(2), 1063–1074.
yī → yí
But he was resigned now. [bʌt hi wəz ɹɪˈzaɪnd ˈnaʊ] natural contour → final rise
Only the final voiced stretch of ‘now’ (at most 400 ms) receives an F0 edit.
Its contour is interpolated in semitones from the natural contour towards a curved rise ending 9 semitones above an anchor taken from the region's first 50 ms. F0 outside the region and the spectral-envelope, timing and voicing controls are held fixed; the demo does not measure statement–question judgements.
natural F0 contour → rising F0 contour
Glottal pulse shape (Rd) and noise gain; formants and F0 unchanged.
ā interjection (啊 ā)
The model's frame-wise Rd control for sustained ā is scaled by factors from 0.6 to 1.6 through a shift on its logarithmic lookup table.
Values are limited to the table's range. F0, formant, noise-branch and duration controls are held fixed; H1–H2 and HNR below are measured outputs, not guaranteed monotonic changes or perceptual labels.
Background on the LF source model. Fant, G. (1995). The LF-model revisited. Transformations and frequency domain analysis. STL-QPSR, 36(2–3), 119–156.
Rd ×0.6 → Rd ×1.6
tā ‘he’ (他 tā)
The gain of ARIS's spectrally shaped noise branch is shifted from −12 to +12 dB for tā.
Harmonic-branch, F0, formant and timing controls are held fixed. HNR is measured on the vowel, alongside pre-voicing energy relative to vowel energy (uv); the manipulation changes noise level rather than aspiration duration.
Evidence that aspiration noise contributes to perceived breathiness. This gain edit does not manipulate voice onset time or establish a consonant category boundary. Klatt, D. H., & Klatt, L. C. (1990). Analysis, synthesis, and perception of voice quality variations among female and male talkers. JASA, 87(2), 820–857.
noise −12 dB → noise +12 dB