C#開源實現MJPEG流傳輸 - gaochundong－IT工程師數位筆記本

文章出處

本文為 Dennis Gao 原創技術文章，發表于博客園博客，未經作者本人允許禁止任何形式的轉載。

許久以前寫了篇文章《基于.NET打造IP智能網絡視頻監控系統》，記錄和介紹了自己幾年來積累和演練的一個系統。發現幾個月過去了，沒有任何進展。

目前已經實現了 UDP+RTP 方式在不同物理機之間的媒體流傳輸。當然，由于沒有基于 .NET 的媒體流壓縮實現，所以直接傳輸的裸圖 Bitmap。不過要求不高，幀率低一些，機器性能強一些，看著也很流暢。

能在桌面客戶端上看到視頻圖像的功能已經完成了。下面需要考慮，如何通過瀏覽器來查看視頻。

在不考慮使用 Flash、ActiveX 的條件下，貌似只能選擇 MJPEG 方式。目前還沒有研究在 HTML5 下視頻是如何處理的，以后有時間可以探索。

什么是 MJPEG？

看這里：

當然，我主要關注 MJPEG over HTTP 這段。

M-JPEG over HTTP
HTTP streaming separates each image into individual HTTP replies on a specified marker. RTP streaming creates packets of a sequence of JPEG images that can be received by clients such as QuickTime or VLC.
In response to a GET request for a MJPEG file or stream, the server streams the sequence of JPEG frames over HTTP. A special mime-type content type multipart/x-mixed-replace;boundary=<boundary-name> informs the client to expect several parts (frames) as an answer delimited by <boundary-name>. This boundary name is expressly disclosed within the MIME-type declaration itself. The TCP connection is not closed as long as the client wants to receive new frames and the server wants to provide new frames. Two basic implementations of a M-JPEG streaming server are cambozola and MJPG-Streamer. The more robust ffmpeg-server also provides M-JPEG streaming support.

也就是說，建立 HTTP 連接后，服務端在 Response 消息中先發一個數據頭 Header 告訴客戶端，我后面的都是 JPEG 圖片。圖片之間使用 boundary-name 來區分，每個圖片前都有自己的數據頭來描述圖片數據長度。

MJPEG數據頭定義

 1     /// <summary>
 2     /// 流頭部
 3     /// </summary>
 4     public string StreamHeader
 5     {
 6       get
 7       {
 8         return "HTTP/1.1 200 OK" +
 9                "\r\n" +
10                "Content-Type: multipart/x-mixed-replace; boundary=" + this.Boundary +
11                "\r\n";
12       }
13     }

 1     /// <summary>
 2     /// 圖片頭部
 3     /// </summary>
 4     public string PayloadHeader
 5     {
 6       get
 7       {
 8         return "\r\n" +
 9                this.Boundary +
10                "\r\n" +
11                "Content-Type: image/jpeg" +
12                "\r\n" +
13                "Content-Length: " + _contentLengthString +
14                "\r\n\r\n";
15       }
16     }

這里的 Boundary 可以是任意字符串，只要你覺得唯一并能區分即可，比如我可以設置為“--dennisgao”。

服務器端實現

Http 服務器其實就是個支持 Tcp 連接的服務器。

1 private AsyncTcpServer _server;
2 
3 _server = new AsyncTcpServer(Port);
4 _server.Encoding = Encoding.ASCII;

 1     public void Start()
 2     {
 3       _server.Start(10);
 4       _server.ClientConnected += new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
 5       _server.ClientDisconnected += new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
 6     }
 7 
 8     public void Stop()
 9     {
10       _server.Stop();
11       _server.ClientConnected -= new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
12       _server.ClientDisconnected -= new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
13     }
14 
15     private void OnClientConnected(object sender, TcpClientConnectedEventArgs e)
16     {
17       _clients.AddOrUpdate(e.TcpClient.Client.RemoteEndPoint.ToString(), e.TcpClient, (n, o) => { return e.TcpClient; });
18     }
19 
20     private void OnClientDisconnected(object sender, TcpClientDisconnectedEventArgs e)
21     {
22       TcpClient clientToBeThrowAway;
23       _clients.TryRemove(e.TcpClient.Client.RemoteEndPoint.ToString(), out clientToBeThrowAway);
24     }

這里可以參考兩篇文章中的實現。

發送圖片數據

首先要保證，對一個HTTP連接只能發一次流頭，因為后面是接連不斷的圖片數據。當然，發點別的數據客戶端也不會解碼。

 1     private void WriteStreamHeader()
 2     {
 3       if (_clients.Count > 0)
 4       {
 5         foreach (var item in _clients)
 6         {
 7           Logger.Debug(string.Format(CultureInfo.InvariantCulture,
 8             "Writing stream header, {0}, {1}{2}", item.Key, Environment.NewLine, StreamHeader));
 9 
10           _server.SyncSend(item.Value, StreamHeader);
11 
12           TcpClient clientToBeThrowAway;
13           _clients.TryRemove(item.Key, out clientToBeThrowAway);
14         }
15       }
16     }

發送圖片數據時，要保證圖片的前面是圖片頭和長度信息，數據尾部要有換行符。

 1     private void WritePayload(byte[] payload)
 2     {
 3       string payloadHeader = this.PayloadHeader.Replace(_contentLengthString, payload.Length.ToString());
 4       string payloadTail = "\r\n";
 5 
 6       Logger.Debug(string.Format(CultureInfo.InvariantCulture,
 7         "Writing payload header, {0}{1}", Environment.NewLine, payloadHeader));
 8 
 9       byte[] payloadHeaderBytes = _server.Encoding.GetBytes(payloadHeader);
10       byte[] payloadTailBytes = _server.Encoding.GetBytes(payloadTail);
11       byte[] packet = new byte[payloadHeaderBytes.Length + payload.Length + payloadTail.Length];
12       Buffer.BlockCopy(payloadHeaderBytes, 0, packet, 0, payloadHeaderBytes.Length);
13       Buffer.BlockCopy(payload, 0, packet, payloadHeaderBytes.Length, payload.Length);
14       Buffer.BlockCopy(payloadTailBytes, 0, packet, payloadHeaderBytes.Length + payload.Length, payloadTailBytes.Length);
15 
16       _server.SendToAll(packet);
17     }

結果演示

在可以成功發送流信息和圖片信息后，就可以在瀏覽器上查看視頻了。當然，我用的 Google Chrome 。IE10 好奇葩，它會把流當成文件不停的下載，搞不懂。

遠程訪問

局域網內的無線設備，只要瀏覽器支持 MJPEG ，均可以查看視頻。我測試了 iPad 上的 Safari 是可以的，但 Chrome 卻直接解析成亂碼。

當然，如果在路由器上配置轉發規則，就可以在外網訪問了。

完整代碼

  1   public class MJpegStreamingServer
  2   {
  3     private static string _contentLengthString = "__PayloadHeaderContentLength__";
  4     private AsyncTcpServer _server;
  5     private ConcurrentDictionary<string, TcpClient> _clients;
  6 
  7     public MJpegStreamingServer(int listenPort)
  8       : this(listenPort, "--dennisgao")
  9     {
 10     }
 11 
 12     public MJpegStreamingServer(int listenPort, string boundary)
 13     {
 14       Port = listenPort;
 15       Boundary = boundary;
 16 
 17       _server = new AsyncTcpServer(Port);
 18       _server.Encoding = Encoding.ASCII;
 19       _clients = new ConcurrentDictionary<string, TcpClient>();
 20     }
 21 
 22     /// <summary>
 23     /// 監聽的端口
 24     /// </summary>
 25     public int Port { get; private set; }
 26 
 27     /// <summary>
 28     /// 分隔符
 29     /// </summary>
 30     public string Boundary { get; private set; }
 31 
 32     /// <summary>
 33     /// 流頭部
 34     /// </summary>
 35     public string StreamHeader
 36     {
 37       get
 38       {
 39         return "HTTP/1.1 200 OK" +
 40                "\r\n" +
 41                "Content-Type: multipart/x-mixed-replace; boundary=" + this.Boundary +
 42                "\r\n";
 43       }
 44     }
 45 
 46     /// <summary>
 47     /// 圖片頭部
 48     /// </summary>
 49     public string PayloadHeader
 50     {
 51       get
 52       {
 53         return "\r\n" +
 54                this.Boundary +
 55                "\r\n" +
 56                "Content-Type: image/jpeg" +
 57                "\r\n" +
 58                "Content-Length: " + _contentLengthString +
 59                "\r\n\r\n";
 60       }
 61     }
 62 
 63     public void Start()
 64     {
 65       _server.Start(10);
 66       _server.ClientConnected += new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
 67       _server.ClientDisconnected += new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
 68     }
 69 
 70     public void Stop()
 71     {
 72       _server.Stop();
 73       _server.ClientConnected -= new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
 74       _server.ClientDisconnected -= new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
 75     }
 76 
 77     private void OnClientConnected(object sender, TcpClientConnectedEventArgs e)
 78     {
 79       _clients.AddOrUpdate(e.TcpClient.Client.RemoteEndPoint.ToString(), e.TcpClient, (n, o) => { return e.TcpClient; });
 80     }
 81 
 82     private void OnClientDisconnected(object sender, TcpClientDisconnectedEventArgs e)
 83     {
 84       TcpClient clientToBeThrowAway;
 85       _clients.TryRemove(e.TcpClient.Client.RemoteEndPoint.ToString(), out clientToBeThrowAway);
 86     }
 87 
 88     public void Write(Image image)
 89     {
 90       if (_server.IsRunning)
 91       {
 92         byte[] payload = BytesOf(image);
 93 
 94         WriteStreamHeader();
 95         WritePayload(payload);
 96       }
 97     }
 98 
 99     private void WriteStreamHeader()
100     {
101       if (_clients.Count > 0)
102       {
103         foreach (var item in _clients)
104         {
105           Logger.Debug(string.Format(CultureInfo.InvariantCulture,
106             "Writing stream header, {0}, {1}{2}", item.Key, Environment.NewLine, StreamHeader));
107 
108           _server.SyncSend(item.Value, StreamHeader);
109 
110           TcpClient clientToBeThrowAway;
111           _clients.TryRemove(item.Key, out clientToBeThrowAway);
112         }
113       }
114     }
115 
116     private void WritePayload(byte[] payload)
117     {
118       string payloadHeader = this.PayloadHeader.Replace(_contentLengthString, payload.Length.ToString());
119       string payloadTail = "\r\n";
120 
121       Logger.Debug(string.Format(CultureInfo.InvariantCulture,
122         "Writing payload header, {0}{1}", Environment.NewLine, payloadHeader));
123 
124       byte[] payloadHeaderBytes = _server.Encoding.GetBytes(payloadHeader);
125       byte[] payloadTailBytes = _server.Encoding.GetBytes(payloadTail);
126       byte[] packet = new byte[payloadHeaderBytes.Length + payload.Length + payloadTail.Length];
127       Buffer.BlockCopy(payloadHeaderBytes, 0, packet, 0, payloadHeaderBytes.Length);
128       Buffer.BlockCopy(payload, 0, packet, payloadHeaderBytes.Length, payload.Length);
129       Buffer.BlockCopy(payloadTailBytes, 0, packet, payloadHeaderBytes.Length + payload.Length, payloadTailBytes.Length);
130 
131       _server.SendToAll(packet);
132     }
133 
134     private byte[] BytesOf(Image image)
135     {
136       MemoryStream ms = new MemoryStream();
137       image.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
138 
139       byte[] payload = ms.ToArray();
140 
141       return payload;
142     }
143   }