want to protect code in atmega but need to update code via bootloader

[vinodstanur]

Hi,
I want to update code in avr using a serial bootloader but at the same time want to protect the code so that no one can read it...
In such case what can I do?

 

[tpetar]

Atmel AVR231: AES Bootloader
http://www.atmel.com/Images/doc2589.pdf



Decryption is the method of retrieving the original message or data and typically cannot be performed without knowing the proper key. Keys can be stored in the bootloader of a microcontroller so that the device can receive encrypted data, decrypt it and reprogram selected parts of the Flash or EEPROM memory. Decryption keys cannot be retrieved from the encrypted data and cannot be read from AVR microcontrollers if lock bits have been programmed accordingly.

 

[vinodstanur]

So, for reading back for the data verification, the microcontroller should send the flash data after encryption, right?
If so, I need to modify the pc side flashing program (say avr-dude) to decrypt the data right?
But what if people hack the decryption code from pc so that they can decode the verification binary sent by the avr to pc?

 

[tpetar]

There is no code verification of protected uC on standard way.

Code is only downloaded in encrypted state from PC to uC, after receiving encrypted code, code is decrypted in AVR inside using decryption keys for this job.

- - - Updated - - -

Some questions on page 13:

What will happen if power is removed during a firmware update?
What is the state of the microcontroller when power is restored back?
Are lock bits and reset vectors set properly at all times?

Are there any assumptions that can be made on what plaintext data will look like?
In order for AES to be broken, there must be a pattern to look for.
The attack software will have to be configured to search for a known pattern, such as interrupt vectors at the start of program memory, memory areas padded with zero or one, and so on.

Is there any feedback that can be derived from the decryption process? Any such feedback can help the attacker. For example, if the decryption algorithm inside the bootloader would give an OK / Not-OK type of signal for each block processed, then this signal could be used as feedback to the attacker.

Should encrypted frames be sent in another order? If the first frame sent to the bootloader always includes the first block of the encrypted file then the attacker can make some assumptions from this. For example, it can be assumed that the first frame maps program data starting from address zero and that it contains the interrupt vector table. This information helps the attacker to refine the key search.

To increase the security of the system, send the frames in random order (the decrypted frames will be mapped to their proper address, anyhow).