SHOUTcast Stream Ripper

Separate metadata from the SHOUTcast stream to automatically name and split the MP3 data and save to disk.

 

Introduction

This article is an extension of the article from Dani Forward. It implements the SHOUTcast protocol to get the metadata header from the SHOUTcast streams and read out the song titles. With this information, it is possible to automatically split the songs, store them as MP3 files on the hard disk and give them the correct song title that comes with the stream.

The source code

This is a 'one lazy night' project and doesn't pretend to be the best implementation of ripping the SHOUTcast stream. But it works simple and fine and may give you an idea, how to get the necessary information from the SHOUTcast stream and how to use them. So, any comments or improvements would be appreciated!

System libraries

First of all, we need the following Libraries:

Hide   Copy Code

using System;
using System.IO;
using System.Net;
using System.Text.RegularExpressions;

Establish the connection to the SHOUTcast Server

Then we establish a connection to the SHOUTcast Server. To get the song titles from the SHOUTcast stream, we need to alter the HttpWebRequest header and add "Icy-MetaData: 1". With that flag set, the SHOUTcast servers will send us the metadata with the song titles, if available.

The stream always begins with the ICY metadata header that comes with the HttpWebResponse object. This header contains some information about the SHOUTcast server and might look like this:

Hide   Copy Code

icy-notice1: This stream requires <A href="http://www.winamp.com/" target=_blank>Winamp</A> 
icy-notice2: SHOUTcast Distributed Network Audio Server/Linux v1.9.5
icy-name: RadioABF.net - Paris Electro Spirit Live From FRANCE
icy-genre: Techno House Electronic
icy-url: <A href="http://www.radioabf.net/" target=blank>http://www.radioabf.net/</A>
content-type: audio/mpeg
icy-pub: 1
icy-metaint: 32768
icy-br: 160

The icy-metaint: 32768 parameter is the most important value for us, because it tells us the blocksize of the MP3 data. In this example, the size of one MP3 block is 32768 bytes. After the connection has been established, the stream starts with an MP3 block of 32768 bytes. This block is followed by one single byte, that indicates the size of the following metadata block. This byte usually has the value 0, because there is no metadata block after each MP3 block. If there is a metadata block, the value of the single byte has to be multiplied by 16 to get the length of the following metadata block. The metadata block is followed by an MP3 block, the single metadata length byte, eventually a metadata block, an MP3 block, and so on (source: Shoutcast Metadata Protocol by Scott McIntyre).

Hide   Shrink   Copy Code

[STAThread]
static void Main()
{
 // examplestream: Radio ABF - http://www.radioabf.net
 // url: http://relay.pandora.radioabf.net:9000

 // station parameters
 String server = "http://relay.pandora.radioabf.net:9000";
 String serverPath = "/";

 String destPath = "C:\\"; // destination path for saved songs

 HttpWebRequest request = null; // web request
 HttpWebResponse response = null; // web response

 int metaInt = 0; // blocksize of mp3 data
 int count = 0; // byte counter
 int metadataLength = 0; // length of metadata header

 // metadata header that contains the actual songtitle
 string metadataHeader = "";
 // last metadata header, to compare with 
 // new header and find next song
 string oldMetadataHeader = null;

 byte[] buffer = new byte[512]; // receive buffer

 // input stream on the webrequest
 Stream socketStream = null;
 // output stream on the destination file
 Stream byteOut = null;

 // create request
 request = (HttpWebRequest) WebRequest.Create(server);

 // clear old request header and build 
 // own header to activate Icy-metadata
 request.Headers.Clear();
 request.Headers.Add("GET", serverPath + " HTTP/1.0");
 // needed to receive metadata informations
 request.Headers.Add("Icy-MetaData", "1");
 request.UserAgent = "WinampMPEG/5.09";

 // execute request
 try
 {
  response = (HttpWebResponse) request.GetResponse();
 }
 catch (Exception ex)
 {
  Console.WriteLine(ex.Message);
  return;
 }

 // read blocksize to find metadata block
 metaInt = Convert.ToInt32(
           response.GetResponseHeader("icy-metaint"));

Receive bytes and separate the metadata from the MP3 data

After the connection to the SHOUTcast server has been established, byte blocks are read from the stream in an endless loop. Every single byte from the MP3 block will be counted and written to the output stream. As long as no metadata block with a song title information has been received, no output file will be created and no MP3 data will be written.

This is an example of a metadata block within the stream:

Hide   Copy Code

StreamTitle=' House Bulldogs - But your love (Radio Edit)';StreamUrl='';

With Regular Expressions it's quite simple to extract the song title from the metadata block. After 32768 bytes have been counted and written, the MP3 block is followed by the metadata length byte. This value is multiplied by 16 and stored in the headerLength integer. If this field is != 0, the metadata header will be written into the metadataHeader string and the headerLength is decremented by 1. When the headerLength field reaches 0, the complete header is written to the metadataHeader string. Now, the metadataHeader string will be compared with the oldMetadataHeader string that stores the last read metadata block. If the new block is not equal to the last block, that means the song has changed. Then, the song title will be extracted from the metadata block, a new file will be created with the extracted title and the output stream will be set to this file. Then, the writing process of MP3 data to the file starts again.

Hide   Shrink   Copy Code

 try
 {
  // open stream on response
  socketStream = response.GetResponseStream();

  // rip stream in an endless loop
  while (true)
  {
   // read byteblock
   int bytes = socketStream.Read(buffer,
                            0, buffer.Length);
   if (bytes < 0)
    return;

   for (int i=0 ; i < bytes ; i++)
   {
    // if there is a header, the 'metadataLength' 
    // would be set to a value != 0. Then we save 
    // the header to a string
    if (metadataLength != 0)
    {
     metadataHeader += Convert.ToChar(buffer[i]);
     metadataLength--;
     // all metadata informations were written 
     // to the 'metadataHeader' string
     if (metadataLength == 0)
     {
      string fileName = "";

      // if songtitle changes, create a new file
      if (!metadataHeader.Equals(oldMetadataHeader))
      {
       // flush and close old byteOut stream
       if (byteOut != null)
       {
        byteOut.Flush();
        byteOut.Close();
       }

       // extract songtitle from metadata header. 
       // Trim was needed, because some stations 
       // don't trim the songtitle
       fileName =
          Regex.Match(metadataHeader,
           "(StreamTitle=')(.*)(';StreamUrl)").Groups[2].Value.Trim();

       // write new songtitle to console for information
       Console.WriteLine(fileName);

       // create new file with the songtitle from 
       // header and set a stream on this file
       byteOut = createNewFile(destPath, fileName);

       // save new header to 'oldMetadataHeader' string, 
       // to compare if there's a new song starting
       oldMetadataHeader = metadataHeader;
      }
      metadataHeader = "";
     }
    }
    // write mp3 data to file or extract metadata headerlength
    else
    {
     if (count++ < metaInt) // write bytes to filestream
     {
      // as long as we don't have a songtitle, 
      // we don't open a new file and don't write any bytes
      if (byteOut != null)
      {
       byteOut.Write(buffer, i, 1);
       if (count%100 == 0)
        byteOut.Flush();
      }
     }
     // get headerlength from lengthbyte and 
     // multiply by 16 to get correct headerlength
     else
     {
      metadataLength = Convert.ToInt32(buffer[i])*16;
      count = 0;
     }
    }
   }
  }
 }
 catch (Exception ex)
 {
  Console.WriteLine(ex.Message);
 }
 finally
 {
  if (byteOut != null)
   byteOut.Close();
  if (socketStream != null)
   socketStream.Close();
 }
}

Method to create a new file and return the stream on this file

The method Main is used to create a new file and return an output stream onto this file. First, the method removes all the characters, that are not allowed in filenames. Then it checks, if the destination folder exists. If not, it will be created. After that, the method checks, if the filename already exists. If it exists, the file will not be overwritten. Instead, a new file with the filename <filename>(i).mp3 will be created.

Hide   Shrink   Copy Code

private static Stream createNewFile(String destPath,
                                        String filename)
{
 // remove characters, that are not allowed 
 // in filenames. (quick and dirrrrrty ;) )
 filename = filename.Replace(":", "");
 filename = filename.Replace("/", "");
 filename = filename.Replace("\\", "");
 filename = filename.Replace("<", "");
 filename = filename.Replace(">", "");
 filename = filename.Replace("|", "");
 filename = filename.Replace("?", "");
 filename = filename.Replace("*", "");
 filename = filename.Replace("\"", "");

 try
 {
  // create directory, if it doesn't exist
  if (!Directory.Exists(destPath))
   Directory.CreateDirectory(destPath);

  // create new file
  if (!File.Exists(destPath + filename + ".mp3"))
  {
   return File.Create(destPath + filename + ".mp3");
  }
  // if file already exists, don't overwrite it. Instead,
  // create a new file named <filename>(i).mp3
  else
  {
   for (int i=1;; i++)
   {
    if (!File.Exists(destPath + filename +
                              "(" + i + ").mp3"))
    {
     return File.Create(destPath + filename +
                               "(" + i + ").mp3");
    }
   }
  }
 }
 catch (IOException)
 {
  return null;
 }
}

Summary

This is a quick example, how to use the metadata in the SHOUTcast streams, to automatically name and split the stream into separate MP3 files. But it is not absolutely 100% safe for all possible streams. For example, if a station doesn't send any track information and the metadata block looks like this: StreamTitle='';StreamUrl='';, the program would split the songs correctly, but would name them ".mp3", "(1).mp3", "(2).mp3" and so on. But I kept it simple to show you just the basics of the protocol.

With some experience about using threads, it is no problem to use this code for downloading multiple streams at the same time. I have added some extra features, like different destination folders for the different streams, storing the stream information in an XML file, a view over all the streams with their status, downloaded bytes, bytes per second, etc. All this together with a GUI gives you quite a nice program to rip multiple streams at the same time. But as I mentioned at the beginning of this article, it was a 'quick and dirty' program that was developed during a lazy nightshift, and it's nothing I would let somebody see the spaghetti-source code of.