Kategorie: Private Programming

My private programming projects

Emulation of a Kawai K1 as VSTi plugin

Introduction

The Kawai K1 is a digital synthesizer made in 1988. It has been used in numerous music productions in the 90s and is even used nowadays by some groups because of its unique sound.

I recently needed some sounds from the K1 family (K1, K1r, K1m, PHm), but for these devices, samples are quite limiting as some of the presets are heavily modulated and depend on things like velocity, aftertouch and so on.

While searching for samples, I stumbled upon a blog post by Chvad, who released the entire set of the K1’s internal 256 waveforms. Part of this set are all the famous PCM samples that I still remember, like the choir, the pan flute and others.
But most of them are not really usable without further editing. Although the K1 is a PCM based sample playback machine (and yes, it is the only form of synthesis its actually doing), Kawais approach for most sounds was the following: Most PCM samples are only very small snippets, they represent only a single cycle of a waveform. They contain the root frequency plus up to 128 harmonics. If this snipped is looped, it creates the sound of a bell, a basic sine wave or others while saving a lot of ROM size, as memory was

I played around with these single cycle waves and was able to create some of the K1 sounds in a sampler. Some of the samples provided by Chvad needed further editing and, due to the frequency they have been sampled at, lacked the high frequencies when playing low Midi note numbers.

This was the first time I thought about if it was possible to emulate a K1 by recreating its internal synthesis. I had this idea in mind for some time, but never had the time to start. During Corona, this idea came back to live again and I started with further research and eventually started this project.

Of course, I know that there are tons of devices & VST plugins out there that have much higher capabilities, may sound better and so on. But as a teenager, the K1 was always one of the devices that I wanted to have, but never bought before I sold all my hardware many years later and moved over to use VST plugins instead.

Reading K1 Sysex data

Luckily, Kawai scanned all their old manuals and put them up on the Internet. This includes the manual itself, the list of wave forms including pictures and the midi implementation. Furthermore, the factory presets and all presets that came on cards are available as sysex files, so that seemed to be a good start.

Reading the sysex files was pretty straightforward. Although the documentation contains some bugs, I quickly figured out the parameters and a comparison between my data structures with a demo version of MidiQuest proved that reading the data is working correctly.

The K1 synthesis engine

The K1 works in the following way:

The K1 features 256 8-bit PCM waveforms. 204 of them are the previously mentioned single cycle waves, the remaining ones are regular PCM samples and feature sounds like choir, flutes, drums, FX and others.

A sound, in K1 terminology they’re called Singles, can be created from up to four so-called Sources. For each source, a arbitrary waveform can be selected. Furthermore, each source has an DADSR envelope (an ADSR with delay), own settings for coarse & fine tune, keytracking can be toggled and there are plenty of other modulation capabilities, both for frequency and amplitude.

It features a single LFO that can either be triangle, square, saw or random that can modulate the frequency of the sources. It is mostly used in combination with the modulation wheel to drive the vibrato function, but also to create surprising modulation effects.

In contrast to the more pricier models a this time, there is neither any filter nor an effects section.

Emulating the first factory patch ‚Aah‘

After I have been able to read presets from sysex files, I’ve set up a VST project and started to emulate it by simply playing back Chvad’s wave files when a midi note is pressed. This turned out to be fairly easy and worked well.

I then added more features to make the first factory preset, called Ahh, playable.

It’s made up of four sources where two of them each have the same waveform with a small detune. After adding support for coarse & fine tune, it sounded better.

I then added support for envelopes by copying some source code from my Heat Synthesizer project and dealt with voice management to make it become a real synth.

I roughly estimated the envelope parameters conversion to make it sound like in this video, which helped me a lot during the first days:

Great success! That sounded like a real K1!

I was surprised how easy it was and continued to implement more features.

Vibrato & Pitch Bend

One of the first things I wanted to add is support for pitch bend and vibrato.

Pitch bend was fairly easy as it was well documented. There is a bend range parameter and the Midi implementation states that 8 bits of the incoming midi data are used. For now, I used the full range of 14 bits for more precision, but I haven’t decided yet if I keep it like this or go down to 8 bits again to match the original machine.

In contrast, implementing the vibrato was much more complex than I initially thought. There is no documentation in the manual that explains the vibrato speed, other than stating that 0 is slowest while 50 is fastest. I roughly estimated the vibrato speed again by listening to the Youtube video that I linked above to make it sound right.

Issues

While I was happy that the first factory preset sounded good, others were completely off. The second factory preset 6 String revealed that my rough envelope estimation was okay for just one preset. The decay & release times were way too long for this one so I reworked my formula and tried to match the original K1 even closer.
But another thing turned out to be problematic: The quality of the sampled waveforms. Especially for 6 String, playing lower notes showed that a lot of upper frequencies are missing. I found a second set of waveforms and played around with them, some of them are much better as they have been sampled at a lower midi note.
Factory preset 5, called Visitors, is the first one that makes heavy use of the LFO to create effects. The LFO timing was a problematic issue for a longer time.

I’m going to post more updates, including audio examples, about the progress and K1 internals in the future. Meanwhile, I luckily got a real K1 and I started to measure a lot of things to make my VST match the real device. Stay tuned & thanks for reading!

Bandwidth metering my ISP: Unitymedia

As my internet line bandwidth is terrible every day during „prime time“ (about between 8pm and 11pm) I wanted to create a small test to have a proof to show to my ISP as well as to find out the critical hours.

As my ISP always tells me to test with a specific speed test (actually it resides on a web page owned by the provider, giving a small distance in hops), I started analyzing what the speed test really does.

The page is http://speedtest-1.unitymedia.de/

As it turns out, the speed test is a flash application. When analyzing the network traffic you quickly find out that the speed test basically does nothing more than downloading a huge 4000 x 4000 pixels random jpg file and then measuring the time it took to download it:

http://speedtest-1.unitymedia.de/speedtest/random4000x4000.jpg?x=randomnumber

The address is called with a parameter called ‚x‘ which is randomized at every new test.

Knowing how to do it, I fired up Notepad++ and created a small script in node.js that repeats this test every two minutes and dumps the result in a syslog file that I can feed into my small Splunk instance to be able to graph it:

var syslog = require('./syslog');
var syslog = require('./syslog');
var http = require('http');

var intervalSeconds = 120;

var totalMBit = 0;
var totalBytesPerSec = 0;
var numRunning = 0;

function bpsToMBit(_bps)
{
	return _bps * 8.0 / (1024 * 1024);
}

function testBandwidthMultiple()
{
	if( numRunning > 0 )
		return;

	totalMBit = 0;

	for( var i=0; i<6; ++i )
	{
		++numRunning;
		testBandwidth();
	}
}
function testBandwidth()
{
//	var url = 'http://speedtest-1.unitymedia.de/speedtest/random4000x4000.jpg';
	var url = 'xxxxx';

	url = url + "?x=" + (Math.random() * 100000.0 + 10000.0);

	var size = 0;

	var t0 = 0;
	var t1 = 0;

	console.log( "Connecting to " + url );

	var request = http.get(url, function(res)
	{
		res.setEncoding('binary')
//		console.log('HEADERS: ' + JSON.stringify(res.headers));

		res.on('end', function()
		{
			var dt = t1 - t0;
			
			var bytesPerSec = size / dt * 1000.0;

			var mbit = bpsToMBit(bytesPerSec);
			
			totalMBit += mbit;
			totalBytesPerSec += bytesPerSec;
			
			--numRunning;
			
			console.log( "size " + size + ", bytes per sec " + bytesPerSec + ", mbit " + mbit + ", " + numRunning + " parts left" );			

			if( numRunning == 0 )
			{
				console.log( "finished, size " + size + ", total bytes per sec " + totalBytesPerSec  + ", total mbit " + totalMBit );

				syslog.log( "BANDWIDTH"	, "isp", "Unitymedia"
										, "mbit", totalMBit
										, "bytespersec", totalBytesPerSec
										, "size", size ); 
			}
		})
		
		res.on('data', function(data)
		{
			var tCurrent = Date.now();

			if( t0 == 0 )
			{
				t0 = tCurrent;
			}
			else
			{
				size += data.length;
			}

			t1 = tCurrent;
		})
		
		res.on('error', function(e)
		{
			console.log("Error: " + e);
		})
	});
}

testBandwidthMultiple();

setInterval( testBandwidthMultiple, 1000 * intervalSeconds);

So while typing this article, the test is now running since some minutes and the first graph can be shown:

unitymedia_bandwidth

Given the fact that I pay for a 100 mbit line, this is terrible. Of course, I’ll post a longer graph tomorrow when the script has been run for a longer time.

Update October 1st, 2015:

So as promised, here is a graph for the last 24 hours:

unitymedia_geschwindigkeit_splunk

As you can see, there is a huge bandwidth drop starting at about 6pm, lasting until about 1am. This also happend one day before.

Additionally, you can see that I never managed to get above 50MBit and I was starting to hunt for a bug in my code but didn’t find one. I modified the code now to use five connections at once (code snipped above is updated), but still there was no large increase. Then suddently the speed test from Unitymedia went down and I was no longer able to test it further so I had to find a new host where I can download test data from.

At first, I tried my small V-Server but as you can see, it couldn’t provide enough bandwidth. But after that, finally I’ve found a new host so the tests can continue. As you can see, I finally reach the promised 100 Mbits so this evening will be the first day where you can see the full bandwidth drop. The next image will be posted tomorrow.

Update October 6th, 2015:

Sorry for being late, but here are the new graphs:

unitymediabandwidth_week unitymediabandwidth_2

As you can see, the problem gets worse on weekends. So far, a Unitymedia guy will be here on Wednesday to check my line. Doesn’t make much sense in my case because the SNR levels on the modem look okay, but anyway he will be there.

High Performance / Low Latency Audio on Android

My thoughts on the current state of low latency audio on Android can be found on the site of my app Heat Synthesizer:

http://heatvst.com/wp/2013/11/30/high-performance-low-latency-audio-on-android-why-it-still-doesnt-work/