How Does A Camera Capture An Image

This topic shows many of the ways one can stream video and/or capture images from the esp32-camera past AIthinker. For this purpose the example "camera_web_server" plant in the esp-who repository (https://github.com/espressif/esp-who.git) was congenital and flashed to a device (use esp-idf v3.two or v3.3 in order to build successfully). The example's main web page is only one of many ways to stream and capture. In the following various approaches are demonstrated. At the end is described how to configure the camera'southward web server to handle WIFI interruptions that can kill the streaming.

Using the Browser

Using VLC

ffmpeg

ffplay

Capture and Display Stream in opencv

Capture Stream and Save to File with opencv

Streaming with Gstreamer

Continuous Streaming Problem Solved

Camera Failure Update

Browser

The relevant code for the browser is establish in app_httpd.c. The default port is port 80 (found in esp_http_server.h). In particular look at app_httpd_main() to find the URI for capturing an image straight.

                      httpd_uri_t capture_uri = {         .uri       = "/capture",         .method    = HTTP_GET,         .handler   = capture_handler,         .user_ctx  = Goose egg     };                  

Therefore, one can capture an image simply by entering http://<your IP>/capture in whatsoever browser. In a similar vein one tin can stream video in any browser. Annotation the URI:

                      httpd_uri_t stream_uri = {         .uri       = "/stream",         .method    = HTTP_GET,         .handler   = stream_handler,         .user_ctx  = NULL     }; ........ config.server_port += 1;                  

Nonetheless, the server port is incremented by ane for streaming. Therefore, entering http://<your IP>:81/stream in a browser will showtime the video stream.

Using VLC

That one now knows the URL it would seem straightforward to employ VLC by inputting the network stream URL. Still, it was institute that only capturing an image ( http://<your IP>/capture ) worked by displaying the image for 10 seconds.

When streaming video was attempted ( http://<your IP>:81/stream ) VLC failed to display it fifty-fifty though Electric current Media Information-Statistics showed that input bytes were streaming. Information technology appears that VLC cannot determine what codec to use. This is a strange effect since browsers easily empathise that TCP packets are beingness sent over HTTPD.

ffmpeg

ffmpeg is used to either capture an paradigm and save it to a file or to stream a video to a file.

          ffmpeg -i http://yourIP/capture -c copy -y my.jpg  ffmpeg -i http://yourIP:81/stream -an -t 30 -c copy -y out.mp4        

ffplay

ffplay essentially uses ffmpeg to stream video to the screen.

          ffplay –i http:/yourIP:81/stream        

Capture and Display Stream with opencv

The post-obit cpp code needs to be built on a machine with opencv installed.

                      #include "opencv2/opencv.hpp"  using namespace cv;  int main(int, char**) {     VideoCapture cap("http://yourIP:81/stream"); // open the default photographic camera     if(!cap.isOpened())  // cheque if nosotros succeeded         return -1;      namedWindow("Stream", CV_WINDOW_AUTOSIZE);      for(;;)     {         Mat frame;         cap >> frame; // get a new frame from camera         // do whatsoever processing         imshow("Stream", frame);         if(waitKey(xxx) >= 0) break;   // you can increase delay to 2 seconds here     }     // the camera will be deinitialized automatically in VideoCapture destructor     return 0; }        

Capture Stream and Save to File with opencv

Example cpp code:

          #include "opencv2/opencv.hpp" #include "iostream" #include "stdio.h" #include "time.h" #include "chrono"  using namespace std; using namespace cv;   int principal(){     // Create a VideoCapture object and use camera to capture the video   VideoCapture cap("http://yourIP:81/stream");      // Check if camera opened successfully   if(!cap.isOpened())   {     cout << "Error opening video stream" << endl;      return -1;    }      // Default resolution of the frame is obtained.The default resolution is system dependent.  //  int frame_width = cap.go(CV_CAP_PROP_FRAME_WIDTH);  //  int frame_height = cap.go(CV_CAP_PROP_FRAME_HEIGHT);     //this returns -one which seems to signal the propid is not supported   cout << "fps = " << cap.go(CV_CAP_PROP_FPS) << endl;    int frame_width = 320;   int frame_height =240;    cap.gear up(CV_CAP_PROP_FRAME_WIDTH,  frame_width);   cap.set(CV_CAP_PROP_FRAME_HEIGHT, frame_height);    printf("width = %d  peak = %d\r\n", frame_width, frame_height);       VideoWriter video("outmjpg.avi",CV_FOURCC('M','J','P','K'),thirty, \ Size(frame_width,frame_height));   auto first = std::chrono::high_resolution_clock::now(); int framecount = 0;   while(1)   {      Mat frame;            framecount = framecount + 1;      // Capture frame-by-frame      cap >> frame;        // If the frame is empty, break immediately     if (frame.empty())       break;           // Write the frame into the file 'outcpp.avi'     video.write(frame);          // Display the resulting frame         imshow( "Frame", frame );        // Press  ESC on keyboard to  go out     char c = (char)waitKey(1);     if( c == 27 )        break;     automobile finish = std::chrono::high_resolution_clock::now();     std::chrono::duration elapsed = finish - kickoff;      if ( elapsed.count() > 10 )     {       cout << "Elapsed time: " << elapsed.count() << " s\n";       cout << "FPS = " << framecount/elapsed.count() << endl;       break;     }   }   cout << "frame count = " << framecount << endl;   // When everything done, release the video capture and write object   cap.release();   video.release();     // Closes all the windows   destroyAllWindows();   return 0; }        

Streaming with Gstreamer

gstreamer is a powerful, simply complicated-to-use, tool. Here it is used to display the stream from the esp32-cam. Presented here is the command that volition role correctly but only later on an important modify is fabricated to the code in app_httpd.c . Offset the command:

          GST_DEBUG=*:iv gst-launch-1.0 -5 souphttpsrc location=http://yourIP:81/stream  ! queue ! \ multipartdemux ! jpegdec ! autovideosink        

The problem arises plain because of the way gstreamer finds the boundary for defining the get-go of the stream of the jpeg image and the terminate of the image. The stream from the webserver is substantially "mjpeg" where the TCP packets bear the image data of each jpeg frame. Below is an instance of what a camera sends at the beginning of a stream of jpeg images:

          HTTP/1.0 200 OK Server: alphapd Date: Mon January  2 02:20:20 2017 Pragma: no-cache Cache-Control: no-cache Content-Type: multipart/x-mixed-supersede;boundary=video boundary-- --video boundary-- Content-length: 12132 Date: 01-02-2017 02:20:20 AM IO_00000000_PT_000_000 Content-type: epitome/jpeg        

Note that the line Content-Type defines the boundary at which a jpeg frame ends and the next one begins. Still also note that this boundary value is repeated on the next line. For some reason gstreamer looks to this 2nd line to find the value of the purlieus, ignoring the definition on the Content-Blazon line. Equally written in app_httpd .c this second line is missing and as a result gstreamer fails with an error message indicating that the purlieus value is missing. Other software manifestly read the Content-Type line. To correct this meet below what needs to exist changed in app_httpd .c in the stream_handler function:

                      static esp_err_t stream_handler(httpd_req_t *req){     camera_fb_t * fb = Naught;     esp_err_t res = ESP_OK;     size_t _jpg_buf_len = 0;     uint8_t * _jpg_buf = Zip;     char * part_buf[64]; #if CONFIG_ESP_FACE_DETECT_ENABLED     dl_matrix3du_t *image_matrix = Zippo;     bool detected = fake;     int face_id = 0;     int64_t fr_start = 0;     int64_t fr_ready = 0;     int64_t fr_face = 0;     int64_t fr_recognize = 0;     int64_t fr_encode = 0; 

Share this post