convert interlaced video into progressive video.

[ombadei]

fpga video processing

Hi,

My team and i are about to embark on a project with the given details below:
- converting a PAL video camera stream into RGB for display on a VGA monitor
- processing the video stream to detect moving objects
- controlling the position of a virtual object (superimposed onto) in the scene by means of the detected motion (eg.of a hand).

The tools we are provided with:
- Digilent Nexys FPGA board (Xilinx XC3S200 Spartan 3)
- Digilent Video Decoder accessory board (ADV7183B decoder chip)
- One Digilent VGA module

I would like to gain some insights on how to deal with this design. We'll be using ISE10 as VHDL development in achieving the output.

To date, i am reading about PAL format and i am quite confused with its structure of YCbCr color space.

As a starting point, i am thinking of laying out the specifications. Might be useful if i can get a kickstart on what factors i should consider.

Appreciate on any comments given.

 

[formjk]

video processing fpga

Hi,

I have been working on something similar application but on an Altera board for sometime.

I can brief about the video pipeline implemented on an altera board. May be you can find something similar for xilinx and can implement it.

first of all, the decoder IC (ADV7183B) will take PAL as input and will give you output in ITU-R BT.656 YCrCb 4:2:2 format, which has YCbCr 4:2:2 format. it means the output pixel is in this format.. Y0 Cb0 .. Y1 Cr1 .. (for each pixel, the Cb and Cr components will be sent alternatively.) . For RGB output, you will need to do color space conversion from YUV to RGB.

the output Bt.656 is interlaced and if you want to display it on VGA monitor, you will need to convert this interlaced video into progressive video.

this video will have resolution of PAL , which need to be converted into VGA resolution which is 640x480. and then you will have this stream ready to be displayed on VGA monitor.

Altera has Video IPs for these different blocks. Not sure about the availability of IPs for similar function for Xilinx.

Hope this would help.

 

[ombadei]

bt656 vga

Thanks.. Was pining for a response for the longest time.. New project members for this course.. Seems like i needed the opinions badly as we are students and relatively new to larger-scale vhdl implementation..

Hi,
first of all, the decoder IC (ADV7183B) will take PAL as input and will give you output in ITU-R BT.656 YCrCb 4:2:2 format, which has YCbCr 4:2:2 format. it means the output pixel is in this format.. Y0 Cb0 .. Y1 Cr1 .. (for each pixel, the Cb and Cr components will be sent alternatively.) . For RGB output, you will need to do color space conversion from YUV to RGB.

According to the BT.656 format sequence.. Would it be possible to ignore one of the luma values (e.g Y1).. and convert to Y0Cb0Cr1 to RGB? Thus, recognizing it as a pixel equivalent.

the output Bt.656 is interlaced and if you want to display it on VGA monitor, you will need to convert this interlaced video into progressive video.

I don't quite understand the meaning of interlace video.

As of now, i trying to piece these information simply. My current approach is to store all the converted values in the onboard RAM and once a frame is completed.. The VGA module (FPGA) will replicate out these values.. Is this too simple-minded?

@ltera has Video IPs for these different blocks. Not sure about the availability of IPs for similar function for Xilinx.

Silly question.. what is IPs?


Thanks a heap..

 

[formjk]

itu-r bt 656 to rgb converter

Hi,

interlaced video is a video used for TV output, in which you send the even numbered lines of video in one frame and odd number lines in another frames and so on. i.e 1 full video frame is divided into two frames each with half no of total lines.

check here https://en.wikipedia.org/wiki/Interlace

before converting the YCbCr 4:2:2 to RGB, you will need to convert this 4:2:2 format to 4:4:4 format.. i.e. if one pixel is (Y0 Cb) and second pixel is (Y1 Cr) in 4:2:2 format , then you will be obtaining 4:4:4 by (Y0 Cb Cb) and (Y1 Cr Cr) as YCbCr components. then you can perform the YCbCr to RGB conversion.

still this is an interlaced video. And if you intend to do the object detection stuff, you need a progressive video, i.e. send full full frame as line no.1 followed by line no. 2 and so on..

The IPs are readily available hardware modules to be used in your design. you can check further on https://www.opencores.org/ for freely available IPs for different modules like VGA_display, de-interlacer.

Well, honestly for student project this is a bit difficult task unless you are given a very long project duration.

all the best.

 

[ombadei]

yuv to rgb vhdl

Thank you. Very relevant information that you have provided me.

On the subject of motion control.. Considering that the background is still and due to noise or imperfections of the ADC (ADV7183).. Are there typical ranges that a pixel value will defer with its introduction of these imperfections? For example, the RGB value of [200,0,0], what values does it deviate to (e.g. [200+Δr, Δg, Δb])?

 

[formjk]

bt 656 vga

I m sorry, I didn't understand your question.

 

[ombadei]

fpga video detection

Ok.. Let me rephrase..

If my video were to capture a red color background, the YCbCr data is converted to its RGB equivalent.

With consideration of noise, i would expect the data value to change. If it changes, is there a typical range that corresponds?

In a nutshell, we have to detect motion and noise is not considered as motion. The monitor will only display when a valid form of motion is detected.

 

[formjk]

fpga video process

U will be performing the YCbCr to RGB conversion in hardware on FPGA.
so either its noise or its valid video pixel, both of them will be having value in fixed range.

e.g. for 24-bit RGB , R, G and B each will be of 8-bit, which means the range will be from 0 to 255.

So by looking at the pixel value, you wont be able to show if it is a valid pixel or a noise.

 

[ombadei]

itu-r bt.656 format

Thanks.. It would seem i can't really determine actual movement versus noise in the background.

For the moment, i will concentrate on just streaming the video. On this factor, we will be using RAM to stored the converted RGB values and reading it to display on monitor. It is quite evident that some form of synchronisation is needed in VHDL as memory can only be accessed at one instance. And the other factor to dwell is that we have interlaced data being 'translated' to progressive.

What do you suggest?

 

[formjk]

itu-r bt656 - rgb

the storing of RGB values for VGA display depends on the availability of large enough memory on your board. I am afraid, i have not worked on xilinx boards so would not be able to guide much on that.

May be you should search for vga display demos or reference design for your xilinx board and then you will have a better idea about the process.

all the best !

 

[f_ris]

fpga video conversion

hi all,

I have the same project that you and I have a small worry:
I don't know how to import some videos that movie the camera so to send some data to the FPGA card

If you have some ideas to propose to me ?

Thank you

Added after 10 minutes:


sorry,
I don't know how to import the videos on my computer: FPGA card ==> my computer

Thank you

 

[ombadei]

view bt656 stream

Do u need to download the stream to the computer (in an O/S environment)?

 

[f_ris]

project itu-r bt.656 decoder xilinx

Frankly, I don't know.

I have a camera connected in the FPGA card and I would like to import videos of camera to my computer and visualize it by a graphic interface but now my problem is to import these videos

Thank you for your answer

 

[ombadei]

rgb out

Sounds like your problem is just an input stream data acquisition project. Once you get all the data downloaded into your pc, i think you will know how handle from there. 'Formjk' gave the insights on the PAL format, so that will be helpful and you can just google to find out.

I think u have to ask yourself these questions:
1) What resources are available to you?
2) What is the PC and FPGA communication (e.g USB)?
3) Video decoder ICs?

My team is using development boards that the school acquired from digilent. The FPGA to PC interface is controlled by the development board's DLL file. We used that for our previous project.

As to date, i have only managed to control the VGA. Still finding a way to read from RAM. I am still rather new to VHDL.

Hope my information is able to help you gain some insights on how you are going to address the issue. Good luck.

 

[f_ris]

fpga video procesiing

thank you for these information, me to I'm using developement boards (VHDL) and I am new to it.

I have a question for the input stream data acquisition project. this is a software that I have to install in my computer ? I looked for it on internet but i have anything found if not i looked for badly!!

can you tell me what is it please!, if have you the time of course.
thank you

 

[ombadei]

bt656 to rgb

if you are using Digilent's development board, then download its software package at http://www.digilentinc.com/Products/Detail.cfm?Prod=ADEPT&NavTop=2&NavSub=69&DetailType= This only provides the basic communication.

For custom built to suite your application, i suppose you have to either make use of its communication protocol or built your own.

 

[ombadei]

fpga make background on vga

For the current stage:

We are going to store the video (PAL-interlace) decoder converted data in the external RAM. But at the same time, i need to be displaying the data on VGA which requires RAM access.

Will it be possible not to display the VGA when the decoder is working. And will this time lapse create any issues?