How to connect VGA CMOS camera sensor to FPGA board

[FlyingDutch]

Hello,

I would like to connect small (VGA) CMOS camera sensor to FPGA board. I had been looking for similiar thread on this forum, but I found only this thread:

https://www.edaboard.com/threads/cmos-camera-connection-to-fpga-de2-board.234189/

Tis thread is related to connecting IP camera to FPGA board, not my case. I would like to connect "OV7670 640x480 VGA CMOS Camera Modul" to Artix-7 FPGA board. My FPGA board is based on " XC7A100T-2FGG676I " Artix-7 FPGA chip:

http://pl.aliexpress.com/item/40001...chweb0_0,searchweb201602_4,searchweb201603_55

I found such OV7670 camera modules in internet shop:

http://www.banggood.com/Wareshare-O...-p-1478355.html?rmmds=search&cur_warehouse=CN

http://www.banggood.com/OV7670-640x...-p-1558348.html?rmmds=search&cur_warehouse=CN

The difference between these two modules is that first module hasn't FIFO buffer chip and second has. Now I am wondering which module to choose that with FIFO bufor or the one without it. I think that this model with FIFO is better because not need big buffer in FPGA internal memory. Could someone to advise me which modue to choose?

I found such projects related to connecting this OV7670 CMOS camera module to FPGA boards:

http://www.dejazzer.com/eigenpi/digital_camera/digital_camera.html

http://www.instructables.com/id/Connect-Camera-to-Zybo-Board/

http://www.fpga4student.com/2018/08/basys-3-fpga-ov7670-camera.html

http://projects.digilentinc.com/cc-ad/cmos-sensor-camera-system-9f74f8

The last projects seems to be most comprehensive and interesting, but also much complex than others from given list.

I will be graceful for any hints how to start with such project.

I am going to start with this project (it seems to me be rudimentary):

http://www.fpga4student.com/2018/08/basys-3-fpga-ov7670-camera.html

Thanks in advance and Regards

 

[KlausST]

Hi,

What I'd do:
* contact the manufacturer
* sign the NDA
* get support (datasheets, application notes)

Klaus

 

[FlyingDutch]

Hi,

What I'd do:
* contact the manufacturer
* sign the NDA
* get support (datasheets, application notes)
it is
Klaus

Hello Klaus,

I am afraid that I wouldn't be able to know manufacturer (this is product from Chinese internet shop). I found the basic datasheet for OV7670 CMOS camera, but I think I can't do more. I think first I will buy the module without FIFO buffer and if it will working with FPGA board then I try the module with FIFO buffer.


BTW: I found on product image (this cameramodule with FIFO buffer) that the FIFO is based on "AVERLOGIC AL422B" chip. Here is link to documentation:

https://docs.rs-online.com/101c/0900766b810246d6.pdf

There are basic information and timing diagrams, I think it is enough information to connect it to FPGA.

Kind Regards

 

[KlausST]

Hi,

Nonsense (excuse me)
Most hits of an internet search tell that the manufacturer is "OmniVision" and even a lot of them lead to their internet site www.ovt.com.

My recommendation: For any electronic part the first place of information should be the manufacturer. They are most reliable, they have the most experience and they provide them for free because the want to sell their parts.

Klaus

 

[FvM]

There are basic information and timing diagrams, I think it is enough information to connect it to FPGA.

I agree. The FIFO version is particularly interesting if your host hardware isn't able to keep up with the OV7670 pixel rate. This problem shouldn't occur with a FPGA system.

 

[fenglei]

Hi,I have done some work on OV7670、OV5640 sensor and both sensor has no buffer.First，if your pcb board has enough interface for sensor module and your pcb layout has done the Signal Integrity problem,the only problem can be coding、simulation and testing.To deal with sensor without buffer ,you will need a fifo in the FPGA hierarchy.
Following is an ov5640 sensor.
**broken link removed**

 

[FlyingDutch]

Hi,I have done some work on OV7670、OV5640 sensor and both sensor has no buffer.First，if your pcb board has enough interface for sensor module and your pcb layout has done the Signal Integrity problem,the only problem can be coding、simulation and testing.To deal with sensor without buffer ,you will need a fifo in the FPGA hierarchy.

Hello,
I had finished that project successfuly some time ago. I used QMTECH Artix7 FPGA board - see link:

QMTECH Artix7 FPGA board

Camera sensor OV7670 is working properly, but with resolution 320x240. I had to modify project a little - constarint file for Artix7 QMTECH FPGA board is"

## Pin assignment
    
set_property CFGBVS VCCO [current_design]
set_property CONFIG_VOLTAGE 3.3 [current_design]
set_property BITSTREAM.CONFIG.UNUSEDPIN PULLUP [current_design]
#############SPI Configurate Setting##################
set_property BITSTREAM.CONFIG.SPI_BUSWIDTH 4 [current_design]
set_property CONFIG_MODE SPIx4 [current_design]
set_property BITSTREAM.CONFIG.CONFIGRATE 50 [current_design]
############## clock and reset define##################
create_clock -period 20.000 [get_ports clk100]
set_property IOSTANDARD LVCMOS33 [get_ports clk100]
set_property PACKAGE_PIN M22 [get_ports clk100]
set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets {Inst_PLL/clkin1}]   

    ##VGA Connector
    set_property PACKAGE_PIN AB26 [get_ports {vga_r[0]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[0]}]
    set_property PACKAGE_PIN AC26 [get_ports {vga_r[1]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[1]}]
    set_property PACKAGE_PIN AB24 [get_ports {vga_r[2]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[2]}]
    set_property PACKAGE_PIN AC24 [get_ports {vga_r[3]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[3]}]
    set_property PACKAGE_PIN AA24 [get_ports {vga_b[0]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[0]}]
    set_property PACKAGE_PIN AB25 [get_ports {vga_b[1]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[1]}]
    set_property PACKAGE_PIN AA22 [get_ports {vga_b[2]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[2]}]
    set_property PACKAGE_PIN AA23 [get_ports {vga_b[3]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[3]}]
    set_property PACKAGE_PIN Y25 [get_ports {vga_g[0]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[0]}]
    set_property PACKAGE_PIN AA25 [get_ports {vga_g[1]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[1]}]
    set_property PACKAGE_PIN W25 [get_ports {vga_g[2]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[2]}]
    set_property PACKAGE_PIN Y26 [get_ports {vga_g[3]}]               
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[3]}]
    set_property PACKAGE_PIN Y22 [get_ports vga_hsync]                       
        set_property IOSTANDARD LVCMOS33 [get_ports vga_hsync]
    set_property PACKAGE_PIN Y23 [get_ports vga_vsync]                       
        set_property IOSTANDARD LVCMOS33 [get_ports vga_vsync]

## LEDs
set_property PACKAGE_PIN J6 [get_ports {config_finished}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {config_finished}]
                    
##Buttons
set_property PACKAGE_PIN H7 [get_ports btnc]                       
    set_property IOSTANDARD LVCMOS33 [get_ports btnc]
set_property PACKAGE_PIN J8 [get_ports btnl]                       
     set_property IOSTANDARD LVCMOS33 [get_ports btnl]
set_property PACKAGE_PIN H21 [get_ports btnr]                       
         set_property IOSTANDARD LVCMOS33 [get_ports btnr]
## OV7670 Camera header pins

##Pmod Header JB
##Sch name = JB1
set_property PACKAGE_PIN W21 [get_ports {ov7670_pwdn}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_pwdn}]
##Sch name = JB2
set_property PACKAGE_PIN Y21 [get_ports {ov7670_data[0]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[0]}]
##Sch name = JB3
set_property PACKAGE_PIN V26 [get_ports {ov7670_data[2]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[2]}]
##Sch name = JB4
set_property PACKAGE_PIN W26 [get_ports {ov7670_data[4]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[4]}]
##Sch name = JB7
set_property PACKAGE_PIN U25 [get_ports {ov7670_reset}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_reset}]
##Sch name = JB8
set_property PACKAGE_PIN U26 [get_ports {ov7670_data[1]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[1]}]
##Sch name = JB9
set_property PACKAGE_PIN V24 [get_ports {ov7670_data[3]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[3]}]
##Sch name = JB10
set_property PACKAGE_PIN W24 [get_ports {ov7670_data[5]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[5]}]
 

##Pmod Header JC
##Sch name = JC1
set_property PACKAGE_PIN V23 [get_ports {ov7670_data[6]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[6]}]
##Sch name = JC2
set_property PACKAGE_PIN V21 [get_ports ov7670_xclk]                   
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_xclk]
##Sch name = JC3
set_property PACKAGE_PIN W23 [get_ports ov7670_href]                   
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_href]
##Sch name = JC4
set_property PACKAGE_PIN V18 [get_ports ov7670_siod]                   
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_siod]
    set_property PULLUP TRUE [get_ports ov7670_siod]
##Sch name = JC7
set_property PACKAGE_PIN W18 [get_ports {ov7670_data[7]}]                   
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[7]}]
##Sch name = JC8
set_property PACKAGE_PIN U22 [get_ports ov7670_pclk]                   
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_pclk]
    set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets {ov7670_pclk_IBUF}]
##Sch name = JC9
set_property PACKAGE_PIN V22 [get_ports ov7670_vsync]                   
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_vsync]
##Sch name = JC10
set_property PACKAGE_PIN U21 [get_ports ov7670_sioc]                   
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_sioc]

In attachement is ziped full project for Vivado 2018.3.(working properly). I have to made VGA DAC based on resitors - here is schematics (get from Basys3 FPGA board):

If you have code in HDL for this (OV7670) or similaiar camera sensor and you would like share it then I will be happy .

Best regards

 

[fenglei]

Hello,
I had finished that project successfuly some time ago. I used QMTECH Artix7 FPGA board - see link:

QMTECH Artix7 FPGA board

Camera sensor OV7670 is working properly, but with resolution 320x240. I had to modify project a little - constarint file for Artix7 QMTECH FPGA board is"

## Pin assignment
 
set_property CFGBVS VCCO [current_design]
set_property CONFIG_VOLTAGE 3.3 [current_design]
set_property BITSTREAM.CONFIG.UNUSEDPIN PULLUP [current_design]
#############SPI Configurate Setting##################
set_property BITSTREAM.CONFIG.SPI_BUSWIDTH 4 [current_design]
set_property CONFIG_MODE SPIx4 [current_design]
set_property BITSTREAM.CONFIG.CONFIGRATE 50 [current_design]
############## clock and reset define##################
create_clock -period 20.000 [get_ports clk100]
set_property IOSTANDARD LVCMOS33 [get_ports clk100]
set_property PACKAGE_PIN M22 [get_ports clk100]
set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets {Inst_PLL/clkin1}]

    ##VGA Connector
    set_property PACKAGE_PIN AB26 [get_ports {vga_r[0]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[0]}]
    set_property PACKAGE_PIN AC26 [get_ports {vga_r[1]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[1]}]
    set_property PACKAGE_PIN AB24 [get_ports {vga_r[2]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[2]}]
    set_property PACKAGE_PIN AC24 [get_ports {vga_r[3]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_r[3]}]
    set_property PACKAGE_PIN AA24 [get_ports {vga_b[0]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[0]}]
    set_property PACKAGE_PIN AB25 [get_ports {vga_b[1]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[1]}]
    set_property PACKAGE_PIN AA22 [get_ports {vga_b[2]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[2]}]
    set_property PACKAGE_PIN AA23 [get_ports {vga_b[3]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_b[3]}]
    set_property PACKAGE_PIN Y25 [get_ports {vga_g[0]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[0]}]
    set_property PACKAGE_PIN AA25 [get_ports {vga_g[1]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[1]}]
    set_property PACKAGE_PIN W25 [get_ports {vga_g[2]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[2]}]
    set_property PACKAGE_PIN Y26 [get_ports {vga_g[3]}]            
        set_property IOSTANDARD LVCMOS33 [get_ports {vga_g[3]}]
    set_property PACKAGE_PIN Y22 [get_ports vga_hsync]                    
        set_property IOSTANDARD LVCMOS33 [get_ports vga_hsync]
    set_property PACKAGE_PIN Y23 [get_ports vga_vsync]                    
        set_property IOSTANDARD LVCMOS33 [get_ports vga_vsync]

## LEDs
set_property PACKAGE_PIN J6 [get_ports {config_finished}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {config_finished}]
                 
##Buttons
set_property PACKAGE_PIN H7 [get_ports btnc]                    
    set_property IOSTANDARD LVCMOS33 [get_ports btnc]
set_property PACKAGE_PIN J8 [get_ports btnl]                    
     set_property IOSTANDARD LVCMOS33 [get_ports btnl]
set_property PACKAGE_PIN H21 [get_ports btnr]                    
         set_property IOSTANDARD LVCMOS33 [get_ports btnr]
## OV7670 Camera header pins

##Pmod Header JB
##Sch name = JB1
set_property PACKAGE_PIN W21 [get_ports {ov7670_pwdn}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_pwdn}]
##Sch name = JB2
set_property PACKAGE_PIN Y21 [get_ports {ov7670_data[0]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[0]}]
##Sch name = JB3
set_property PACKAGE_PIN V26 [get_ports {ov7670_data[2]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[2]}]
##Sch name = JB4
set_property PACKAGE_PIN W26 [get_ports {ov7670_data[4]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[4]}]
##Sch name = JB7
set_property PACKAGE_PIN U25 [get_ports {ov7670_reset}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_reset}]
##Sch name = JB8
set_property PACKAGE_PIN U26 [get_ports {ov7670_data[1]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[1]}]
##Sch name = JB9
set_property PACKAGE_PIN V24 [get_ports {ov7670_data[3]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[3]}]
##Sch name = JB10
set_property PACKAGE_PIN W24 [get_ports {ov7670_data[5]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[5]}]


##Pmod Header JC
##Sch name = JC1
set_property PACKAGE_PIN V23 [get_ports {ov7670_data[6]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[6]}]
##Sch name = JC2
set_property PACKAGE_PIN V21 [get_ports ov7670_xclk]                
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_xclk]
##Sch name = JC3
set_property PACKAGE_PIN W23 [get_ports ov7670_href]                
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_href]
##Sch name = JC4
set_property PACKAGE_PIN V18 [get_ports ov7670_siod]                
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_siod]
    set_property PULLUP TRUE [get_ports ov7670_siod]
##Sch name = JC7
set_property PACKAGE_PIN W18 [get_ports {ov7670_data[7]}]                
    set_property IOSTANDARD LVCMOS33 [get_ports {ov7670_data[7]}]
##Sch name = JC8
set_property PACKAGE_PIN U22 [get_ports ov7670_pclk]                
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_pclk]
    set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets {ov7670_pclk_IBUF}]
##Sch name = JC9
set_property PACKAGE_PIN V22 [get_ports ov7670_vsync]                
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_vsync]
##Sch name = JC10
set_property PACKAGE_PIN U21 [get_ports ov7670_sioc]                
    set_property IOSTANDARD LVCMOS33 [get_ports ov7670_sioc]

In attachement is ziped full project for Vivado 2018.3.(working properly). I have to made VGA DAC based on resitors - here is schematics (get from Basys3 FPGA board):


View attachment 163197

If you have code in HDL for this (OV7670) or similaiar camera sensor and you would like share it then I will be happy .

Best regards

Hi,
The following file is found on the internet.And it involved configuration to ov7670 sensor and data receiving from sensor.The problem in image scale may refers to configuration.
At last,do you used the software weak pull-up on the SDA signal.

 

[FlyingDutch]

Hello,

I will look into files that you attached to your post this weekend. The smaller resolution in project I implemented had been intended (on original Basys3 FPGA board there was to little BRAM memory for full VGA buffer and only every fourth pixel was captured). ON my QMTECH FPGA board is enough BRAM memory to implement full VGA frame-buffer and I am going to change resolution to higher in the near future

Best Regards

 

[FlyingDutch]

Hello,

after I finished that simple project described above (connecting OV7670 camera sensor to QMTECH Artix7 FPGA) I decided to implement a little bit more advanced image processing - edge detection. I choosed project from archive of Digilentic.com - here is link to this project:

http://projects.digilentinc.com/cc-ad/cmos-sensor-camera-system-9f74f8

Project originally had been implemented on "Nexys Video" board from Digilentic. I used for it's implementation the same QMTECH FPGA board with Artix7 described in this thread above (it had enough Block RAM to implement full VGA frame-buffer).

I had to made two minor changes: add one "Clock wizard" IPcore (oryginally Nexys board had 100 MHz clock and QMTECH 50 MHz), and negate two reset signals in top.v file (lines 70 and 71). After looking in "Nexys Video" board schematics I noticed that push buttons has inverse logic comparing to QMTech board.

After that project hadn't errors in synthessis and implementation - see scrennshot:

Basically project is working properly on FPGA board - edge detection is working, but screen smudges a bit. It might to be caused by the fact that I attached HDMI output (FPGA board) to very small 10 inch monitor (it hasn't big quality). Here are two pictures of working system:

I am attaching also full Vivado 2020.1 project (for QMTECH Artix7 FPGA) - maybe someone would be interested.
CMOSCameraArtix7My.zip

MY conclusion after implementig these two project is: even for small VGA camera sensor is needed big amount of internal FPGA BRAM memory. It caused that relatively big FPGAs are needed, and not much Logic blocks or flip-flops are used but almos all block RAM from FPGA chip. Currently my next task is to implement RGB frame-buffer in external "DDR SDRAM" memory chip connectd to FPGA chip. I am owner of simple "Mimas v.2" FPGA development board which has on-board Micron "MT46H32M15LF" DDR SDRAM memory (organisation of this RAM is: 32Meg. x 16). Ultimately I am going to be able to run these projects on these two cheap FPGA boards:

http://numato.com/product/mimas-v2-spartan-6-fpga-development-board-with-ddr-sdram

http://pl.aliexpress.com/item/1000006703110.html?spm=a2g17.12010612.8148356.2.4b9c22d4DTSPgo

if someone of forums users have any hints or points them will be warmly welcome

Best Regards

 

[FlyingDutch]

Hello again,

during I had been looking for better camera sensor for my experiments (with FPGA), I found such two binocular (stereo) camera sensors:

https://www.banggood.com/OV2640-Bin...548019.html?rmmds=mywishlist&cur_warehouse=CN

https://www.banggood.com/Sipeed-2-M...469504.html?rmmds=mywishlist&cur_warehouse=CN

Does somebody know valuable links with theory related to "stereo vision" and maybe projects using such sensors (not only based on FPGA but also based on micro-controlers)? I am especially interested in making 3D map of seen scene (in context of robot sight).

Thanks in advance.

 

[BradtheRad]

theory related to "stereo vision"

Left and right images see roughly the same background. Near objects are placed differently in the left view as compared to the right. Humans and animals have nerve pathways that evaluate stereo vision automatically. Computers require a lot of extra effort.

Write software that erases background, so that near objects remain. A crude tactic is to invert colors in one channel, then sum both channels together. Background is pretty much erased.

Is it a high priority to avoid collisions? Shine a dim led of some distinctive color (or infra-red) ahead of the robot. If your camera detects an object lit up in that color, then it's getting close.