+ Post New Thread
Results 1 to 14 of 14
  1. #1
    Member level 2
    Points: 309, Level: 3

    Join Date
    Oct 2018
    Posts
    47
    Helped
    1 / 1
    Points
    309
    Level
    3

    Correct design to work with MIPI and SPI

    Hello.

    I don't know if this is the correct forum to ask about this topic, sorry if I'm wrong.
    I want to connect a camera and a Lattice FPGA. It will be through the MIPI interface. From the FPGA I want to do a couple of things. I would like to connect it to an ARM processor and also to a screen, and what I was asked for was connect it using SPI.
    I guess we can use a MIPI screen and solve the problem, but, my question here is the following. MIPI uses a Parallel bus, and SPI is serial, is it posible to make a "MIPI to SPI"? Another thing I thought was using a shared RAM memory between FPGA and ARM. So I would save the video on the memory and the ARM would access that memory and make an image processing. I don't know if that would be possible or if it is too complicated. What do you think, guys?

    As always, thanks in advance.

    •   AltAdvertisment

        
       

  2. #2
    Super Moderator
    Points: 71,047, Level: 65
    Achievements:
    7 years registered
    Awards:
    Most Frequent Poster 3rd Helpful Member

    Join Date
    Apr 2014
    Posts
    14,528
    Helped
    3306 / 3306
    Points
    71,047
    Level
    65

    Re: Correct design to work with MIPI and SPI

    Hi,

    First things first.

    Draw a sketch. Even hand drawn is OK. Draw the ARM, RAM, FPGA, camera. Draw the busses and name them.
    Also do some basic calculations like (camera) frame rate, X and Y resolution, color depth --> bit rate. --> RAM size.

    You say "video" and "processing". What do you want to process? Do some basic research about algorithms and their ressources (processing power, RAM, busses, throughput...)
    Write down all these requirements on the sketch.

    Do a little research on your own:
    MIPI uses a Parallel bus
    I donīt know about MIPI, but a 10 seconds internet search leads me to MIPI.ORG. There they write:
    The MIPI Display Serial Interface (MIPI DSISM) defines a high-speed serial interface between a host processor and a display module.
    Or do you mean "MIPI PTI"? --> then use correct terminology to avoid confusion.

    Search the internet for similar projects. Look for design descriptions with pitfalls and how they solved it.

    Klaus
    Please donīt contact me via PM, because there is no time to respond to them. No friend requests. Thank you.



  3. #3
    Member level 2
    Points: 309, Level: 3

    Join Date
    Oct 2018
    Posts
    47
    Helped
    1 / 1
    Points
    309
    Level
    3

    Re: Correct design to work with MIPI and SPI

    Basically, this would be the scheme I want to achieve.
    Click image for larger version. 

Name:	MIPI-SPI.png 
Views:	4 
Size:	59.5 KB 
ID:	150432
    I have no specifications from the camera nor the screens. And I also haven't been informed about the finallity of the project.
    The camera goes through MIPI and CSI-2 specification, that's all what I know.

    On the Internet I have found SPI to MIPI and MIPI to N MIPI.

    The FPGA to use is a Lattice ice40, which hasn't an integrated CPU. So we must connect it to an ARM somehow, I have been told to investigate to do it in SPI, but I'm not sure that's possible.



  4. #4
    Super Moderator
    Points: 71,047, Level: 65
    Achievements:
    7 years registered
    Awards:
    Most Frequent Poster 3rd Helpful Member

    Join Date
    Apr 2014
    Posts
    14,528
    Helped
    3306 / 3306
    Points
    71,047
    Level
    65

    Re: Correct design to work with MIPI and SPI

    Hi,

    mysterious.
    No specification? No resolution, no frame rate, nothing...

    This does not sound like a serious assignment.

    ****
    This rather sounds like someone asks you to transport some unknown thing from house X to house Y, with the only requirement to use a rubber raft.
    Not knowing if this thing is data, fluid, gas, solid... not knowing size and weight, not knowing where house X is and if there is water inbetween house X and house Y.

    Wow. I wouldn't like to be in your shoes.

    Klaus
    Please donīt contact me via PM, because there is no time to respond to them. No friend requests. Thank you.



  5. #5
    Member level 5
    Points: 749, Level: 5

    Join Date
    Dec 2017
    Location
    Bydgoszcz - Poland
    Posts
    91
    Helped
    17 / 17
    Points
    749
    Level
    5

    Re: Correct design to work with MIPI and SPI

    Quote Originally Posted by Ironlord View Post
    Basically, this would be the scheme I want to achieve.
    Click image for larger version. 

Name:	MIPI-SPI.png 
Views:	4 
Size:	59.5 KB 
ID:	150432
    I have no specifications from the camera nor the screens. And I also haven't been informed about the finallity of the project.
    The camera goes through MIPI and CSI-2 specification, that's all what I know.

    On the Internet I have found SPI to MIPI and MIPI to N MIPI.

    The FPGA to use is a Lattice ice40, which hasn't an integrated CPU. So we must connect it to an ARM somehow, I have been told to investigate to do it in SPI, but I'm not sure that's possible.

    Hello,

    the FPGA board you mention is not proper for video procesing (not enough resources) - even simple like for example edge detection (with use of Sobel filter). You need SoC FPGA (CPU+FPGA fabric) best with MIPI interface for camera. For example "Zybo Z7":

    https://reference.digilentinc.com/re.../zybo-z7/start

    With this board "Zybo" you can use "Vivado HLS" for converting C/C++ algorithm (for example "Sobel filter") into HDL language, and optimize it for paraleling. Then you can write program for ARM CPU (for aquisition of image/video and put it together with Soble filter on FPGA fabric).

    Regards



  6. #6
    Member level 5
    Points: 749, Level: 5

    Join Date
    Dec 2017
    Location
    Bydgoszcz - Poland
    Posts
    91
    Helped
    17 / 17
    Points
    749
    Level
    5

    Re: Correct design to work with MIPI and SPI

    Hello,

    also I am not sure if for Lattice FPGAs is available software tool similiar to "Vivado HLS" which is able to convert algorithm written in C language to one of HDL languages like Verilog or VHDL.

    Regards



    •   AltAdvertisment

        
       

  7. #7
    Member level 2
    Points: 309, Level: 3

    Join Date
    Oct 2018
    Posts
    47
    Helped
    1 / 1
    Points
    309
    Level
    3

    Re: Correct design to work with MIPI and SPI

    Hello again.

    Click image for larger version. 

Name:	scheme-mipi-to-spi.png 
Views:	3 
Size:	151.8 KB 
ID:	150448
    In the image from above, Lattice guys are making a SPI to MIPI converter.
    What we are looking for id for the inverse operation.

    I have found the module MIPI DSI RX, wich would get the MIPI signal: http://www.latticesemi.com/dsirx
    Also the pixel to byte converter, but I think I can't connect both of them, their output/input seems quite different: http://www.latticesemi.com/Products/...oByteConverter
    And the Compress module, it is missing I guess, so I think I can't do the operation.

    Quote Originally Posted by FlyingDutch View Post
    Hello,

    the FPGA board you mention is not proper for video procesing (not enough resources) - even simple like for example edge detection (with use of Sobel filter). You need SoC FPGA (CPU+FPGA fabric) best with MIPI interface for camera. For example "Zybo Z7":

    https://reference.digilentinc.com/re.../zybo-z7/start

    With this board "Zybo" you can use "Vivado HLS" for converting C/C++ algorithm (for example "Sobel filter") into HDL language, and optimize it for paraleling. Then you can write program for ARM CPU (for aquisition of image/video and put it together with Soble filter on FPGA fabric).

    Regards
    About what you comment, I'm not sure yet if is possible or not, but I would say we need a better FPGA. Lattice also has their own IDE and IP Cores.



  8. #8
    Super Moderator
    Points: 252,897, Level: 100
    Awards:
    1st Helpful Member

    Join Date
    Jan 2008
    Location
    Bochum, Germany
    Posts
    44,056
    Helped
    13404 / 13404
    Points
    252,897
    Level
    100

    Re: Correct design to work with MIPI and SPI

    Did you notice that the MIPI DSI (display serial interface) discussed in post #7 has nothing in common with post #3 MIPI CSI? MIPI doesn't refer to a specific interface standard. The only common point is that both are operating at gigabit/s speed and can't be simply translated to slow serial interfaces like SPI without compressing the workload somehow, e.g. grabbing a still picture from the camera input.



  9. #9
    Member level 5
    Points: 749, Level: 5

    Join Date
    Dec 2017
    Location
    Bydgoszcz - Poland
    Posts
    91
    Helped
    17 / 17
    Points
    749
    Level
    5

    Re: Correct design to work with MIPI and SPI

    Quote Originally Posted by Ironlord View Post
    Hello again.

    Click image for larger version. 

Name:	scheme-mipi-to-spi.png 
Views:	3 
Size:	151.8 KB 
ID:	150448
    In the image from above, Lattice guys are making a SPI to MIPI converter.
    What we are looking for id for the inverse operation.

    I have found the module MIPI DSI RX, wich would get the MIPI signal: http://www.latticesemi.com/dsirx
    Also the pixel to byte converter, but I think I can't connect both of them, their output/input seems quite different: http://www.latticesemi.com/Products/...oByteConverter
    And the Compress module, it is missing I guess, so I think I can't do the operation.



    About what you comment, I'm not sure yet if is possible or not, but I would say we need a better FPGA. Lattice also has their own IDE and IP Cores.
    Hello,

    theoretically you can build design described in your post: separate FPGA and CPU ARM, but it will be extremally hard to program. Also take into consideration that most "Computer Vision" algorithms are written in C/C++ and you have to translate them into one of HDL languages. Has this IDE for Lattice FPGAs such feature like translating C programs into HDL language? Second question is that FPGA board you had chosen has to little resources for video processing.

    Why to complicate task that you can solve in simple way?

    Regards



  10. #10
    Member level 2
    Points: 309, Level: 3

    Join Date
    Oct 2018
    Posts
    47
    Helped
    1 / 1
    Points
    309
    Level
    3

    Re: Correct design to work with MIPI and SPI

    That was also my first thoght.
    But I was told to investigate about it. I was given the scheme I showed you and was told to find a solution for the oposite case.

    I know that I am going from Gigabit/s to 10MB/s, so from the beggning it doesn't seem a good idea to go for SPI. Anyway, is it possible to make the opposite as I explained? In case it isn't, what would be the best idea?

    To program it in C, probably we will end using Matlab HDL Coder or a similar plug in.



    •   AltAdvertisment

        
       

  11. #11
    Member level 5
    Points: 749, Level: 5

    Join Date
    Dec 2017
    Location
    Bydgoszcz - Poland
    Posts
    91
    Helped
    17 / 17
    Points
    749
    Level
    5

    Re: Correct design to work with MIPI and SPI

    Hello,

    if you can use MATLAB it solves the problem with translating CV algorithms into HDL. I am not able to answer your first question - it require some investigation and calculations.

    I wish good luck with your project.

    Regards



  12. #12
    Super Moderator
    Points: 252,897, Level: 100
    Awards:
    1st Helpful Member

    Join Date
    Jan 2008
    Location
    Bochum, Germany
    Posts
    44,056
    Helped
    13404 / 13404
    Points
    252,897
    Level
    100

    Re: Correct design to work with MIPI and SPI

    I know that I am going from Gigabit/s to 10MB/s, so from the beggning it doesn't seem a good idea to go for SPI. Anyway, is it possible to make the opposite as I explained? In case it isn't, what would be the best idea?
    Of course it's possible to connect a camera and a display as sketched. The interesting question is, what do you want to display? You can't send real time video through SPI. It's all about the intended image processing functionality of your design, not the interfaces.



  13. #13
    Member level 2
    Points: 309, Level: 3

    Join Date
    Oct 2018
    Posts
    47
    Helped
    1 / 1
    Points
    309
    Level
    3

    Re: Correct design to work with MIPI and SPI

    I guess I will proposse this design in order to match the specifications.
    Click image for larger version. 

Name:	MIPI-SPI.png 
Views:	3 
Size:	13.8 KB 
ID:	150455
    I have seen the DSIRX and pixel-to-byte IP cores are available. The Decompress/Compress module seems to be the same, but changing the configuration does the one thing or the other.
    I guess the Compress module saves in memory the Byte, and that's what I send to the SPI module, which will put it on the MOSI line, am I right?

    On the other hand, I think I should talk about the posibility of doing all the process working only with MIPI, and in a FPGA with an ARM included.
    Anyway, the iCE40 seems good for video porpousals, what's your opinion about it?

    I know that if you don't know about the purpouse of the project tou can't tell me if that FPGA is enough or not, but neither I know.



  14. #14
    Super Moderator
    Points: 71,047, Level: 65
    Achievements:
    7 years registered
    Awards:
    Most Frequent Poster 3rd Helpful Member

    Join Date
    Apr 2014
    Posts
    14,528
    Helped
    3306 / 3306
    Points
    71,047
    Level
    65

    Re: Correct design to work with MIPI and SPI

    Hi,

    in order to match the specifications.
    What specifications?
    So you have some specifications but donīt tell us?

    Klaus
    Please donīt contact me via PM, because there is no time to respond to them. No friend requests. Thank you.



--[[ ]]--