System level verification

Hi guys!

I want you to share your expriences about system level verification.
Suppose that we have some modules that have been tested individualy
and they all work fine.
Now we integrate them together and we want to check the whole
system. What should we care about?
If you have been in this situation, what was your problems? how
did you solve them? What is your suggestions, ideas and .... ?

Thanks

System level verification begins after all blocks are integrated, assuming all sub-blocks are functionally correct.
Focusing on bugs in:
- connectivity
- block interaction
- control-flow
Efficient testbench architecture can greatly improce verification effectiveness & usability.

For black-box verification
- focuses on functional requirements
- find incorrect or missing functions, interface errors and performance or through put errors

For white-box verification
- focuses on RTL design structures
- derive test cases to ensure all independent paths are exercised at all once
- logical decisions are exercised for both true & false paths
- loops are executed at their boundaries and within operational bounds
- internal logic is exercised to ensure validity

Hope this helps.

I think that system level verification tool must support VHDL,Verilog,C,SystemC...modulers, and offer a lots of function block. For communication, synop*sys 's css is useful for this case. see their web.

Are you referring to CoCentric System Studio? Yup, it is a very useful tool for system level design but one thing to note though, you would have to learn up SystemC. If you have a programming background, it shouldn't be any problem at all. After all, it's an open source language.

Thanks for your replys

As a case study;
suppose that we have some modules
and their models in verilog language,
and we have already tested each of
them individually and they didn't have
any mismatch.
Now, we link all the DUTs together
and all their related models together
and test them again and we
are geting some mismatch (in the
output signals of modules and their
models)
1- How did it come?
2- Is this a resonable way for testing
the system level?
3- how can we test the coverage?
4- How should we define some
assertions and how these assertions
are necessary?

Take care

I recommend SystemC..

May refer to **broken link removed**.

This site consists of korean but you can read english pages.

Hi Cool Member,
Let me share my thoughts and notions on this topic.
First of all, I think the method you are following to bring up the top level testcases ( or testbench whatever it is) is not the correct method of verification.
In System Level verification, there are two things to be noted, one is whether the Design Under Test (as an entity which is top of all modules) is meeting the expected specifications and another thing is whether the testcases are exercising most of the lines or not.

So, in system level verification we should think testing from the Boundary of the Design Entity (which is the top integrated module) and our testcases should be in such a way that they should exercise all lines of code which in turn entire functionality of the core.

Finally, I think bringing up the top testbench from individual testbench (which includes models created at unit level testing) may not be an efficient method of verification.

Any more discussion on this is welcome.

Regards,
- satya

Hi satya,

Thanks for your comments,
Does the top level testbench include a behavioral model of top level DUT?
if we want to model the top level, in which level should we model it?
behavioral? RTL? ...? How can we make sure that we have exercised all
internal paths and functionalities when we just consider all of linked
modules as an one entity(Top Module)?

Another question:

What is your idea about using a special tool or language such as specman
or vera instead of using verilog and its common simulators for verification
In my opinion, if you want to model a design, it makes no difference that
you use verilog or another lang. to do it. But if you want to generate a
directed random flow, a language like 'e' is more powerful than verilog.
(I do not know about SystemC or Vera) What do you think?


Regards

A good design process is to have two teams, one that designs the RTL code and one that designs the test cases at the top level (Component interface). Both teams use the hardware specification as a reference.

The design team make some validation at the block level to validate the basic functionality and the corner cases that are not obvious at the system level.

It is important for the test team to have a test plan that describe the test cases relative to each features of the component. It is always a good idea to implement DFT facilities in the design to improve testability.

Specman is a very good langage for system level validation. It include a powerfull list manipulation commands, randomisation and coverage support.

Regards,
shell3

As you all know, driving a subsystem to a certain state ( which is a corner case) from the Top module's inputs is very difficult and sometimes it
seems impossible. So what should we do for this?

I agree with shell3.
Normally 60% of the project schedule time will be taken by verification. Once at the Functional Specification is prepared for a core, I think both design team and verification team should work in parallell. The Functional Specification should be in such a way that, a Test Engineer who prepares testcases should be able to get a clear cut idea for what functionality he has to prepare testcases to test with.

Normally designer will do basic lelver testing, which we call it as Unit Level testing or whitebox verification.

Hope this helps.
Regards,
- satya

CoolMember said:

Does anybody know about formal verification and semi-formal verification?

Click to expand...

There are several formal verification language....such OVA, OVL, PSL, and assertion of system verilog
I think PSL is the most popular...
if you want to understand PSL and system verilog , you can go to
https://www.accellera.org/ and download LRM.

CoolMember said:

Thanks for your replys

As a case study;
suppose that we have some modules
and their models in verilog language,
and we have already tested each of
them individually and they didn't have
any mismatch.
Now, we link all the DUTs together
and all their related models together
and test them again and we
are geting some mismatch (in the
output signals of modules and their
models)
1- How did it come?
2- Is this a resonable way for testing
the system level?
3- how can we test the coverage?
4- How should we define some
assertions and how these assertions
are necessary?

Take care

Click to expand...

There exists some un-clear points in your description. PLease describe
your methodology. And,

What do you mean "testing"? main function or test coverage?

Roughly,

0)
Usually, we use "divide & conquer" way to verify your design. That means, you must write your golden model first in TOP level. In that , we
define the function for each subblock. So, our test for each subblock is coming from that golden model test patterns. If you already did by this way but still had problem, check the writing way for each block( usually need debugging), some coding style will make simulation algorithm outputs different scheduling event, and these different scheduling events make your result different.

1) If you use "bottom-up" solution, it means you write sub-block first, then merge them do chip level verification, of course it didn't work because you didn't consider from system when you made these sub-block.

2). Assertion in system: you can use tools like vera or something else, but I think simple is the best, in HDL you still can do this if you put self test/checking logic in your design.

In my thought, system level verification is not difficult, not depend on tools so seriously, only depends on how you run the project, then you make decision.

1- For a complex system, do we need a behavioral model for the whole system?
if so, how this model can help us? how should we use it?

2- How can we make sure that we've finished the verification successfully?
can somebody show me a good flowchart for complex system level verification
from the begining to the end?

Behavior models are useful in represent analog block, otherwise, we have to verify complex systems in mixed-mode environment, not a good idea for always tight schedule.

What do you mean by mixed mode environment jimi ?

About functional coverage items: you need to have different functional coverage items for block-level and for top-level verification, because goals of those two verifications are different. On top-level, submodule interface is not controllable and as on block-level verification, so sometimes we could not fully test submodules from top-level.
Verification is generally process that is never really finished. Manager need to decide when it is time to stop further verification because potential benefits are less then costs.
For using of assertions in verification process, please consult the book “Assertion based Design” from Harry Foster.

Where can I find the “Assertion based Design” from Harry Foster?

I think you should deferentiate the chip level & system level verification, and set different goals on these two level verification