SitePoint Sponsor

User Tag List

Results 1 to 5 of 5
  1. #1
    Non-Member bronze trophy
    Join Date
    Nov 2009
    Location
    Keene, NH
    Posts
    3,760
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)

    Question about HTML5 Audio from .js

    I'm trying to make a 16 operator additive synth using just audio for a game I'm slowly putting together... In general I'm finding the implementation for the audio tag a bit... lacking in capabilities for serious use on things like this.

    The whole concept is to have EVERYTHING from the graphics to the audio 100% generated by the javascript.

    My first idea was to simply loop a generated track... I'm generating it thus:

    Code:
    function makeWavFile(data,sampleRate,bitDepth,channels) {
    var
    	byteDepth=Math.floor(bitDepth/8),
    	numChannels=channels,
    	calcSize=data.length*numChannels*byteDepth;
    	var dataStream=(
    		// RIFF HEADER
    		'RIFF' +                      // ChunkID, should always be 'RIFF'
    		dwordToString4(calcSize+36) +   // ChunkSize
    		'WAVE' +                      // Format, should always be 'WAVE'
    		// Format Subchunk
    		'fmt ' +                                          // SubChunk1ID, should always be 'fmt ' -- that extra space is important!
    		dwordToString4(16) +                                // SubChunk1Size, should always be 16 for PCM
    		wordToString2(1) +                                 // AudioFormat 1=PCM
    		wordToString2(numChannels) +                       // NumChannels
    		dwordToString4(sampleRate) +                        // SampleRate
    		dwordToString4(sampleRate*numChannels*byteDepth) +  // ByteRate
    		wordToString2(numChannels*byteDepth) +             // BlockAlign
    		wordToString2(bitDepth) +                          // BitsPerSample
    		// Data SubChunk
    		'data' +                      // SubChunk2ID, should always be 'data'
    		dwordToString4(calcSize)        // SubChunk2Size, aka actual DATA size
    	);
    
    	switch (bitDepth) {
    		case 16:
    			dataStream+=stringEncodeWords(data);
    		break;
    		case 8:
    			dataStream+=stringEncodeBytes(data);
    		break;
    	}
    
    	ccopy=dataStream;
    
    	return 'data:audio/wav;base64,'+btoa(dataStream);
    }
    Which is working fine in FF and Opera for creating my audio... but the problems are two-fold. The first problem is that Chrome doesn't support RIFF (audio/wav) as a audio type NOR does it appear to even support generated content... So, does anyone have an idea on how I can take my raw script generated datastream and make webkit play it?

    The second problem is that, well...

    Code:
    	var noisePlayer=document.createElement('audio');
    	noisePlayer.controls=true;
    	noisePlayer.autoPlay=false;
    	noisePlayer.loop=true;
    	noisePlayer.src=noise;
    	document.body.appendChild(noisePlayer);
    Doesn't automatically loop. I'm able to trap the end and loop back, but in Opera there's this 200ms+ delay that's unacceptable for what I'm using it for. In FF there's no delay.

    I did discover that if I track position and force the play position to zero Opera has no delay, while Gecko does.

    The audio being generated is all going to be flat predictable waveforms, so overbuffering and just looping a small inner section would work fine -- if I can figure out how to make gecko and opera behave.

    Gotta say, for VIDEO they're 'close' to matching flash's capabilities, but for AUDIO this is a ***** joke.

    Been in "GIS mode" for the better part of a week on this, most of the examples online so far seem sketchy at best, overcomplicated train wrecks at worst... It shouldn't be THIS difficult to do something so simple.

    Oh, semi-unrelated, I tossed up the graphics side of things for people to have a look-see.

    http://canvastest.comoj.com/

    To sum up my questions:

    1) script generated sound in Chrome -- possible?

    2) delays in looping in Opera/FF, am I stuck browser sniffing?

    3) is there any DECENT documentation in plain English that actually lists the javascript properties for the HTML AUDIO tag? What I'm coming across is either gibberish (W3C) or code without explanations (mozilla's page on the subject)

    I feel like when I first started dealing with HTML before I discovered the WDG reference... or when I started with javascript before I picked up the old O'Reilly pocket reference... I already know how to program this ****, I just need a decent function reference!

  2. #2
    I meant that to happen silver trophybronze trophy Raffles's Avatar
    Join Date
    Sep 2005
    Location
    Tanzania
    Posts
    4,662
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    It looks like audio codec support is far from uniform. Look at this table: http://html5doctor.com/native-audio-in-the-browser/

    Not sure what to do about that. I suppose Flash is out of the question.

    Possible solution to looping problem: have two audio elements. When one finishes, you immediately start the other. There probably will be a delay, but you never know. Otherwise, I guess you're stuck sniffing for window.opera.

    Regarding the documentation, besides looking at what's available (which I know isn't very good), you could create an audio element, then inspect it in Firebug and click the DOM tab. Then you can see many (if not all) the methods and properties the element supports.

  3. #3
    Non-Member bronze trophy
    Join Date
    Nov 2009
    Location
    Keene, NH
    Posts
    3,760
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    I tried the two alternating - the delay is still there (unacceptably so for my purposes) as is the normal looping and in FF as it turns out. I dropped my sample to 100hz and even when it 'works' there's just too much delay for me to use it.

    Looks like AUDIO is a useless joke -- Far, far way away from being useful for my purposes. Even the delay to start playing audio that you've preloaded takes too long for anything realtime.

    Serious dealbreaker and enough to make you go "Ok, back to flash"

    At this point I'm half tempted to see what I can do with MIDI instead.

  4. #4
    Non-Member bronze trophy
    Join Date
    Nov 2009
    Location
    Keene, NH
    Posts
    3,760
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    Gah, I just tried controlling midi one note at a time, javascript just isn't fast/accurate enough to keep up with even eighth notes at 4/4 120bpm.

    I guess that's why all the 'AUDIO TAG' drum synths I found on google sound like they're being played by Steve Martin's character in "The Jerk" in terms of their timing. (stumbling about on every loop by as much as a quarter-beat!)

    Well, that kinda kills HTML5 for game writing.

  5. #5
    Non-Member bronze trophy
    Join Date
    Nov 2009
    Location
    Keene, NH
    Posts
    3,760
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    Wow, gets worse the more I look into it -- and I thought SDL was bad when it came to latency. I mean, I know javascript is slow, but lands sake this is basically stuff I was doing in Basic on a 1.89mhz 6809 two and a half decades ago.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •