After we gained some experiences with the OpenMAX interface API we began to work on the audio part of the MPEG-TS stream. Basically it is the same procedure that we used for the video data. First we convert the TS data into PS and then the PS data into ES which can be used by the OpenMAX routines. Since the our mpeg utility code from dvbstream can be used for both types of data (audio and video) we just need to provide additional callbacks for the audio data.
Again, we used the hello_pi code examples, more precisely the file audio.c, for our proof-of-concept prototype. The aim was to demultiplex a complete MPEG-TS stream into audio and video, the video data is ignored for the moment, to decode the data on the fly and to pass it to the audio device via the OpenMAX interface. That worked surprisingly well, at least for our very simple environment that only used the audio jack in 2 channel stereo mode.
So, let us check our inventory what we got:
- A module to decode MPEG-TS streams into elementary video and audio data
- A module to utilize the MPEG decoder of the GPU to handle the video data
- A module to decode the audio data
- A module to process IR events issued by an IR receiver
Now, we just need to glue all these components together by defining some simple interfaces and to add some missing features. For instance, to synchronize audio and video streams.