Welcome to the complete guide for the ESP32-Cam! In this post I go over basics, how to use, pins, wiring, programming, taking pictures, uploading to a local sever and even performing computer vision analysis on another device.
The ESP32-Cam is a small camera module that runs on an ESP32-S chip and uses a OV2640 camera. The ESP32_Cam can also the OV7670 camera, but the OV2640 is better (higher resolution and built-in JPEG encode, which removes the processing task from the ESP32-S).
ESP-32 Cam Specifications
ESP-32
It is Wi-Fi capable (802.11b/g/n)
Bluetooth capable (4.2 with BLE)
Built-in LED Flash
9 IO ports
Supports UART, SPI, I2C and PWM
Built-in micro SD card reader
Input power: 3.3V / 5V (it’s reported that powering with 5V is more stable than 3.3V)
OV2640 Camera
2 Megapixels
Array size: UXGA (1600 x 1200)
Lense Size: 1/4″ (6.35mm)
Max image transfer rate: 15 frames/sec
ESP32-Cam Pins
Power In: 5V or 3.3V (5V is recommended!)
GND: Ground
VCC: Outputs 3.3V
IO Pin #16: Can be used without issue for GPIO
IO Pins #1 & #3: Used as UART interface. Since the ESP32-Cam doesn’t have a USB port, these are used to push the code into it. These cannot be used for anything else.
Other IO Pins: Are also used for other purposes (micro SD Card pins, some by WiFi driver, etc) , so it is likely you’ll have issues if you try to use them (specially during boot).
ESP32-Cam Connections for programming
As mentioned, the ESP32-Cam does not have a built-in USB port (which helps keep the size small), as a result we have to use an FTDI programmer to program our ESP32-Cam. Wire the ESP32-Cam as shown on the image and then connect the FTDI as to your computer via USB to push the code.
*Note that we connect IO# 0 with GND because this pin MUST be Low during flashing.
Make sure to set the FTDI jumper to the correct setting. For the setup above, you need it set to 5V.
ESP32-Cam Components Front/Back
As you can see on the two images below, the front of the ESP32-Cam contains the camera connector terminal, an LED Flash, and a microSD Card socket. On the back we have the built-in antenna, an optional external antenna connector, and a reset button. Note that the pins are on the back, so if you place the ESP32-Cam on a solderless breadboard, you will be unable to access the reset button.
Front
Back
Programming ESP32-Cam in Arduino
Adding ESP32 JSON file link to Arduino IDE
Using the ESP32-Cam with Arduino requires adding the appropriate JSON file link within Arduino IDE. This is done in Arduino IDE by going to File –> Preferences. Within the settings tab, towards the bottom you’ll see “Additional Boards Manager URLs”. Mine currently only has the JSON file link for the ESP8266 boards, which I have previously used in other projects. Now I need to add the link for ESP32-Cam. Click the box shown in the image below and a pop-op window will open where you can enter the URL for the JSON file. Since I have more than one URL I separate them with a comma and enter them in separate rows, then click “OK”. See images below (click images to enlarge them):
JSON File Link: https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json
Adding ESP32 board to Arduino IDE
Once the correct JSON file link has been added to Arduino IDE, you need to select the appropriate board in the Arduino IDE Board Manager. The first step is to add the ESP32 boards to the Arduino IDE Board Manager. Do do this go to Tools –> Board –> Board Manager (see images below). Once the pop-op window opens, type ESP32 to search for the board, then install the ESP32 one by Espressif Systems. Depending on your download speed, the installation could take a while.
Once installed, go to Tools –> Board –> ESP32 Arduino –> AI Thinker ESP32-CAM. Note that the list of boards is long, so scroll down the list carefully to not miss it. See screenshot below.
ESP32-Cam Examples
Sending video to your computer
As mentioned previously, you will need to use an FTDI programmer to program our ESP32-Cam. So, make sure to have an FTDI programmer available and setup as shown on the image.
Next, after connecting the ESP32-Cam to the FTDI programmer and the FTDI programmer to your computer (via USB cable) make sure to select the correct Port (Tools –> Port), then go to Arduino IDE and go to File –> Examples –> ESP32 –> Camera –> CameraWebServer. You will see the Arduino code shown below:
As you can see, towards the beginning of the code, there are sections to select your camera model and to enter your WiFi credentials.
For camera model, let’s pick “CAMERA_MODEL_AI_THINKER“, so remove the “//” to uncomment this row and add “//’ to comment the other models.
For the WiFi credentials, enter your WiFi network name inside the quotations next to “ssid”, and enter your password inside the quotations next to “password”.
Next, click the “Upload” button in Arduino IDE (circle with arrow pointing to the right). Once the code is done uploading, disconnect the IO#0 pin from the GND pin on your ESP32-Cam. You only need them connected during code upload.
Next, open the serial monitor and ensure the baud rate is set to 115200, then press the reset button on the back of your ESP32-Cam. You will see text printed on the serial monitor and there should be a line stating “Camera Ready! Use…” and will give you the IP address to connect.
Access video coming from ESP32-Cam
Go to your browser, and enter the address that was provided in our Arduino IDE Serial Monitor. From the previous image, you can see mine was “http://192.168.1.244”, so I copied and pasted that into my browser. Once you go to the IP address provided, you will see a user interface. Step one is clicking “Start Stream”. Once you do, you’ll see the image coming from your ESP-32 Cam! See mine below.
As you can see, you are also able to edit many parameters such as the resolution, brightness, contrast, saturation, etc. You are also able to mirror horizontally or flip vertically. There are also Face Detection/Recognition features.
Take Picture and save on SD Card
In this example we will be taking a picture with our ESP32-Cam and saving it to a microSD Card (using the built-in microSD Card slot). Note that the ESP32-Cam is rated to work with SD cards of up to 4Gb in capacity, although it has been reported to work up to 16Gb. The program will have the ESP32-Cam rest in sleep mode, we will then wake it up via Reset button, then a picture will be taken, and the picture will be saved on the microSD Card.
Formatting the microSD Card
Step 1 in this example is to take our microSD card and format it to FAT32. To do so connect the SD card to your computer, go to “This PC” and right click on the icon for the SD card. Click on the option to format the SD Card. You will see a pop-up window with options. Under file system, make sure to select FAT32 and select the option for “Quick Format” under Format options, then click Start. You will see a warning stating you will lose all data in the SD card if you format it. If this is not an issue, proceed by clicking OK. Wait, a little, and then your microSD card will be formatted.
Once you have successfully formatted your microSD card, go to Arduino IDE and copy/paste the code below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
#include "esp_camera.h" #include "Arduino.h" #include "FS.h" // SD Card ESP32 #include "SD_MMC.h" // SD Card ESP32 #include "soc/soc.h" // Disable brownour problems #include "soc/rtc_cntl_reg.h" // Disable brownour problems #include "driver/rtc_io.h" #include <EEPROM.h> // read and write from flash memory // define the number of bytes you want to access #define EEPROM_SIZE 1 // Pin definition for CAMERA_MODEL_AI_THINKER #define PWDN_GPIO_NUM 32 #define RESET_GPIO_NUM -1 #define XCLK_GPIO_NUM 0 #define SIOD_GPIO_NUM 26 #define SIOC_GPIO_NUM 27 #define Y9_GPIO_NUM 35 #define Y8_GPIO_NUM 34 #define Y7_GPIO_NUM 39 #define Y6_GPIO_NUM 36 #define Y5_GPIO_NUM 21 #define Y4_GPIO_NUM 19 #define Y3_GPIO_NUM 18 #define Y2_GPIO_NUM 5 #define VSYNC_GPIO_NUM 25 #define HREF_GPIO_NUM 23 #define PCLK_GPIO_NUM 22 int pictureNumber = 0; void setup() { WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector Serial.begin(115200); //Serial.setDebugOutput(true); //Serial.println(); camera_config_t config; config.ledc_channel = LEDC_CHANNEL_0; config.ledc_timer = LEDC_TIMER_0; config.pin_d0 = Y2_GPIO_NUM; config.pin_d1 = Y3_GPIO_NUM; config.pin_d2 = Y4_GPIO_NUM; config.pin_d3 = Y5_GPIO_NUM; config.pin_d4 = Y6_GPIO_NUM; config.pin_d5 = Y7_GPIO_NUM; config.pin_d6 = Y8_GPIO_NUM; config.pin_d7 = Y9_GPIO_NUM; config.pin_xclk = XCLK_GPIO_NUM; config.pin_pclk = PCLK_GPIO_NUM; config.pin_vsync = VSYNC_GPIO_NUM; config.pin_href = HREF_GPIO_NUM; config.pin_sscb_sda = SIOD_GPIO_NUM; config.pin_sscb_scl = SIOC_GPIO_NUM; config.pin_pwdn = PWDN_GPIO_NUM; config.pin_reset = RESET_GPIO_NUM; config.xclk_freq_hz = 20000000; config.pixel_format = PIXFORMAT_JPEG; if(psramFound()){ config.frame_size = FRAMESIZE_UXGA; // FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA config.jpeg_quality = 10; config.fb_count = 2; } else { config.frame_size = FRAMESIZE_SVGA; config.jpeg_quality = 12; config.fb_count = 1; } // Init Camera esp_err_t err = esp_camera_init(&config); if (err != ESP_OK) { Serial.printf("Camera init failed with error 0x%x", err); return; } //Serial.println("Starting SD Card"); if(!SD_MMC.begin()){ Serial.println("SD Card Mount Failed"); return; } uint8_t cardType = SD_MMC.cardType(); if(cardType == CARD_NONE){ Serial.println("No SD Card attached"); return; } camera_fb_t * fb = NULL; // Take Picture with Camera fb = esp_camera_fb_get(); delay(1000);//This is key to avoid an issue with the image being very dark and green. If needed adjust total delay time. fb = esp_camera_fb_get(); if(!fb) { Serial.println("Camera capture failed"); return; } // initialize EEPROM with predefined size EEPROM.begin(EEPROM_SIZE); pictureNumber = EEPROM.read(0) + 1; // Path where new picture will be saved in SD Card String path = "/picture" + String(pictureNumber) +".jpg"; fs::FS &fs = SD_MMC; Serial.printf("Picture file name: %s\n", path.c_str()); File file = fs.open(path.c_str(), FILE_WRITE); if(!file){ Serial.println("Failed to open file in writing mode"); } else { file.write(fb->buf, fb->len); // payload (image), payload length Serial.printf("Saved file to path: %s\n", path.c_str()); EEPROM.write(0, pictureNumber); EEPROM.commit(); } file.close(); esp_camera_fb_return(fb); // Turns off the ESP32-CAM white on-board LED (flash) connected to GPIO 4 pinMode(4, OUTPUT); digitalWrite(4, LOW); rtc_gpio_hold_en(GPIO_NUM_4); esp_deep_sleep_start(); } void loop() { } |
Reminders
Remember that the ESP32-Cam must we wired differently for programming vs. normal operation:
Programming
Normal Operation
Once you have ensured your setup is ready for programming, go ahead and click the “Upload” button in Arduino IDE (circle with arrow pointing to the right). Once the code is done uploading, disconnect the IO#0 pin from the GND pin on your ESP32-Cam. Of course, you don’t have to use the FTDI for normal operation. You could power with another power source, and you also don’t need the yellow and green cables, but at a minimum you need to disconnect IO#0 from GND.
Testing
After programming and disconnecting IO#0 from GND, power up your ESP32-Cam and take a few pictures by pushing the built-in reset button. Once you are done taking pictures, power off the ESP32-Cam, remove the microSD card, and insert into your computer. Use a USB adapter if needed. Check the file and you should see files with names as “image1”, “image2”, “image3”, and so on.
Take Picture when PIR Motion Sensor is triggered and save on SD Card
This example is the same as the one above, except now the picture will be taken whenever a PIR sensor detects motion. You can substitute this with any other sensor that basically provides a HIGH whenever you want the picture to be taken (it could even be an external switch). For this example we will use a PIR sensor to mimic a security camera. For more details on how to use a PIR sensor, check out this post.
Wiring our system
Before we jump into this, there is something critical to take into account, and is that the ESP32-Cam already uses a bunch of the IO pins for the camera and for the SD card, so things get tough when we choose to add another IO into the mix. Additionally, from the previous code, remember that we are basically taking a picture and sending the ESP32-Cam into deep sleep. We then wake it up to take another picture, and then back to sleep! So, previously we were using the reset button to wake it up, but we are now using an external signal. So, what pins can we use?
When the ESP32-Cam is in deep sleep, only the low power portion of the system is running (which is great for power savings!). As a result, we can only use IO pins 2, 4, 12, 13, 14, or 15 to wake it up. Additionally, I mentioned the camera and SD card use many pins, so whichever pin we select needs to be free when we are not trying to reset the ESP32-Cam. To do this, we will add an 2N2222 transistor (see this post) as shown below. The goal will be to link our IO pin to ground when we want to reset the ESP32-Cam (to take the picture), and then isolate it at all other times. I picked IO Pin 13 after trying others without luck.
As a tried to test multiple pins I tested the system using a push button, and I then moved into using the actual PIR sensor. See below for details on both.
Programming
Normal Operation
With Pushbutton (for testing):
With PIR:
Arduino Code
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 |
#include "esp_camera.h" #include "Arduino.h" #include "FS.h" // SD Card ESP32 #include "SD_MMC.h" // SD Card ESP32 #include "soc/soc.h" // Disable brownour problems #include "soc/rtc_cntl_reg.h" // Disable brownour problems #include "driver/rtc_io.h" #include <EEPROM.h> // read and write from flash memory // define the number of bytes you want to access #define EEPROM_SIZE 1 RTC_DATA_ATTR int bootCount = 0; // Pin definition for CAMERA_MODEL_AI_THINKER #define PWDN_GPIO_NUM 32 #define RESET_GPIO_NUM -1 #define XCLK_GPIO_NUM 0 #define SIOD_GPIO_NUM 26 #define SIOC_GPIO_NUM 27 #define Y9_GPIO_NUM 35 #define Y8_GPIO_NUM 34 #define Y7_GPIO_NUM 39 #define Y6_GPIO_NUM 36 #define Y5_GPIO_NUM 21 #define Y4_GPIO_NUM 19 #define Y3_GPIO_NUM 18 #define Y2_GPIO_NUM 5 #define VSYNC_GPIO_NUM 25 #define HREF_GPIO_NUM 23 #define PCLK_GPIO_NUM 22 int pictureNumber = 0; void setup() { WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector Serial.begin(115200); Serial.setDebugOutput(true); camera_config_t config; config.ledc_channel = LEDC_CHANNEL_0; config.ledc_timer = LEDC_TIMER_0; config.pin_d0 = Y2_GPIO_NUM; config.pin_d1 = Y3_GPIO_NUM; config.pin_d2 = Y4_GPIO_NUM; config.pin_d3 = Y5_GPIO_NUM; config.pin_d4 = Y6_GPIO_NUM; config.pin_d5 = Y7_GPIO_NUM; config.pin_d6 = Y8_GPIO_NUM; config.pin_d7 = Y9_GPIO_NUM; config.pin_xclk = XCLK_GPIO_NUM; config.pin_pclk = PCLK_GPIO_NUM; config.pin_vsync = VSYNC_GPIO_NUM; config.pin_href = HREF_GPIO_NUM; config.pin_sscb_sda = SIOD_GPIO_NUM; config.pin_sscb_scl = SIOC_GPIO_NUM; config.pin_pwdn = PWDN_GPIO_NUM; config.pin_reset = RESET_GPIO_NUM; config.xclk_freq_hz = 20000000; config.pixel_format = PIXFORMAT_JPEG; pinMode(4, INPUT); digitalWrite(4, LOW); rtc_gpio_hold_dis(GPIO_NUM_4); if(psramFound()){ config.frame_size = FRAMESIZE_UXGA; // FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA config.jpeg_quality = 10; config.fb_count = 2; } else { config.frame_size = FRAMESIZE_SVGA; config.jpeg_quality = 12; config.fb_count = 1; } // Init Camera esp_err_t err = esp_camera_init(&config); if (err != ESP_OK) { Serial.printf("Camera init failed with error 0x%x", err); return; } Serial.println("Starting SD Card"); delay(500); if(!SD_MMC.begin()){ Serial.println("SD Card Mount Failed"); //return; } uint8_t cardType = SD_MMC.cardType(); if(cardType == CARD_NONE){ Serial.println("No SD Card attached"); return; } camera_fb_t * fb = NULL; // Take Picture with Camera fb = esp_camera_fb_get(); delay(1000);//This is key to avoid an issue with the image being very dark and green. If needed adjust total delay time. fb = esp_camera_fb_get(); if(!fb) { Serial.println("Camera capture failed"); return; } // initialize EEPROM with predefined size EEPROM.begin(EEPROM_SIZE); pictureNumber = EEPROM.read(0) + 1; // Path where new picture will be saved in SD Card String path = "/picture" + String(pictureNumber) +".jpg"; fs::FS &fs = SD_MMC; Serial.printf("Picture file name: %s\n", path.c_str()); File file = fs.open(path.c_str(), FILE_WRITE); if(!file){ Serial.println("Failed to open file in writing mode"); } else { file.write(fb->buf, fb->len); // payload (image), payload length Serial.printf("Saved file to path: %s\n", path.c_str()); EEPROM.write(0, pictureNumber); EEPROM.commit(); } file.close(); esp_camera_fb_return(fb); delay(1000); // Turns off the ESP32-CAM white on-board LED (flash) connected to GPIO 4 pinMode(4, OUTPUT); digitalWrite(4, LOW); rtc_gpio_hold_en(GPIO_NUM_4); esp_sleep_enable_ext0_wakeup(GPIO_NUM_13, 0); delay(500); esp_deep_sleep_start(); Serial.println("This will never be printed"); } void loop() { } |
Results from testing (push button and PIR Sensor)
ESP32-Cam with Vision Processing with OpenCV
In this example, we will re-use the example on sending video to your computer (1st example we covered), and then we’ll take the video and process it through Python using OpenCV.
Step 1 will be to download Anaconda, then we’ll download Python within Anaconda and go from there.
To download Anaconda, go to www.anaconda.com, specifically to the link below:
www.anaconda.com/products/distribution#downloads
From there download the appropriate file for your computer. I will install the one for Windows:
Once Anaconda is installed, open Anaconda Powershell Prompt. In there, go ahead and enter the following:
Conda create –name virtualend python=3.11
This will lead to several packages being installed:
When asked if you want to proceed, enter “y”.
Once fully downloaded and installed, it will indicate those processes are done, and will display a message that to activate the virtual environment you will need to type and enter “conda activate virtualenv” and to deactivate an active environment you will need to type and enter “conda deactivate“.
Next, go ahead and enter the virtual environment by typing and entering “conda activate virtualenv“, then type “pip install opencv-contrib-python“. This installation might take a while to be completed.
Next, type and enter “pip install requests“:
Next, type and enter “pip install cvlib“:
Next, type and enter “pip install tensorflow“
Next, type and enter “pip install matplotlib“
Once all of those are installed, take the python code below, and save it on a folder in your computer.
Python Code for Example
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
import cv2 import numpy as np import requests import time import queue import threading import cvlib as cv import matplotlib.pyplot as plt from cvlib.object_detection import draw_bbox class VideoCapture: def __init__(self, name): self.cap = cv2.VideoCapture(name) self.q = Queue.Queue() t = threading.Thread(target=self._reader) t.daemon = True t.start() def _reader(self): while True: ret, frame = self.cap.read() if not ret: break if not self.q.empty(): try: self.q.get_nowait() except Queue.Empty: pass self.q.put(frame) def read(self): return self.q.get() URL = "http://192.168.1.244" cap = cv2.VideoCapture(URL + ":81/stream") if __name__ == '__main__': requests.get(URL + "/control?var=framesize&val={}".format(8)) while True: if cap.isOpened(): ret, frame = cap.read() cv2.imshow("Output", frame) #imgnp=np.array(bytearray(cap.read()),dtype=np.uint8) #im = cv2.imdecode(imgnp,-1) bbox, label, conf = cv.detect_common_objects(frame) im = draw_bbox(frame, bbox, label, conf) cv2.imshow('Output',im) key = cv2.waitKey(3) if key == 27: break cv2.destroyAllWindows() cap.release() |
Testing the example
To test the example follow these steps:
- Complete all steps listed under example #1 (Sending video to your computer), but do not connect to the ESP32-Cam via computer web browser.
- Open Anacoda (if you closed it) and activate the virtual environment via “conda activate virtualenv” if not active already.
- Finally, type “Python ” followed by the file path where you saved the Python code shown above. For me this would be “Python C:\Users\danie\Downloads\Test.py“.
Next, a window will open and video live video will start being processed with computer vision.
Thanks for the really good examples of how to code some of the camera functions. I looked everywhere on the web and this is the best page I could find. Great work!
Which data is being used for the image classification