How to store data on SD card in other format than binary?

[Deepak350]

Hi guys,

I'm starting a project on SD card. I got an example program and its working fine. I am able to write and read data from the SD card. The problem is that the data stored and read is in binary format. (eg. if I write "8" it is stored as "1000" on the card). Is there anyway I can store it in any other formats (like .xml)??
I'm using a 2GB SD card, PIC18F8520 MCU and a MicroC IDE.

Regards,
Tech350.

 

[banjo]

writing txt file on sd card with microcontroller

Data is always stored in binary format. If an XML file is written to the card it still in a binary format. It is the upper software layers that interpret the data into another format. You wrote an "8" and it was stored as "00001000". If you want the ASCII value of the number eight, you have to store hex "38" or binary "00111000". It is the upper layer of software that converts the format to its binary equivalent.

Do a web search for FAT file system. There are some examples of the FAT file system on SD cards. This is what you need to write an XML file to the card with perhaps a PC and then read it back with the microcontroller or vice versa.

Here is a starting point:

**broken link removed**

 

[wizpic]

msp430 sd card fat32

Proton plus has got a libay you create text file, and CSV for excel, so if you wrote 8 it would store it has 8

sorry I have not used MicroC
wizpic

 

[Deepak350]

sd card sector read spi corrupt

Thanks a lot for the reply guys. Well there is a lot of confusions. I'l get my basics right about file systems and get back

 

[nikhilele]

microchip sd card fat minimal ram

if you want to write only one file at a time no need to go in details of fat and making system complicated.. you can write file directly by making your own software layer
viz converting full file in words of 8bit and write is to card and while reading remake the file again..

 

[Deepak350]

read sd card sector

Thanks for the reply nikhilele,

Okay I'm starting the SD project from scratch. I want to initialize the SD in FAT16 file system. How can I go about that? Well I got an example code and its working fine. I can successfully write data into the SD card, but when I want to view the files or data in my computer it is in an unknown format. So I wanted to format or initialize the SD in FAT16 format.
Is it possible?
What is the sequence?
Is there an example code in CD?

Thanks in advance

 

[Deepak350]

re initialize a sd card

Hello,

Can anybody give me the SD/MMC initialization sequence or logic please?

Regards,
Deepak.

 

[banjo]

to detect and initialize an sd card only

I would use your computer to format the SD card. Why write microcode to do a job the computer is already good at!

Once you have the card re-formatted, use the computer to copy a single small text file to the card.

Next, write some microcode that opens this text file from the SD card and appends some data to the end. Close the file and then remove the card and put it back into the computer card reader. Use the PC tools to check the card for disk errors and then open the text file and check if your new data was appended. Once you can modify the file on your PIC without causing a disk error, you got all the major issues sorted out. When the first couple of tries corrupt the SD card, just use the computer to reformat it and try again.

By doing the above, you make the computer do the initial FAT setup for adding a new file, this way your microcode does not have to do it and the code can be smaller. For most embedded projects, this is not a big limitation. For embedded stuff, the required file names are known well in advance.

 

[Deepak350]

how to find if sd card has mbr

Hi banjo thanks for your reply, in order to append the files in the SD card (after formatting in the computer, etc.) we have to initialize the SD card in the first place. On doing so the SD will be initialized and formatted in a format unknown. The SD card has to be initialized by you before you want to perform any other functions on it. This is my opinion and experience, please correct me if I'm wrong.

 

[banjo]

sd card unknown format

OK, it seems I misinterpret what you were saying. In my way of thinking, the PC initializes the SD card and the PIC only reads back this initialization. Regardless, here are the steps as I understand them.

First, take a look at :
http://en.wikipedia.org/wiki/File_Allocation_Table

Skip the beginning part, it is both confusing and not necessary. Move down to the Design portion and look at diagram of the disk partition. This shows how data is grouped on the partition. Each section is a known size. The FAT or File Allocation Table is a series of linked lists that allow you to piece the file back together. Special entries in the FAT help find unused clusters and the EOC which marks the end of that particular file.
While the FAT helps you sequentially find all the pieces of the file, how to you find the starting point? The answer is the root directory section. This again is at a known location in the partition. You go to the root directory and search for the file name you are interested in. Once the file name is matched up, the data within this root directory entry will tell you the starting point of the file. Then you go to that location in the FAT to start piecing the file back together.
When using a microcontroller, the job can be simplified, if you are willing to make some compromises. If the card only contains only a few files and you know the order they were created in, then you can precalculate the location of the locations in the root directory structure where the first FAT cluster will be. This keeps you from having to include code to search the root directory. Other simplications are possible, if you always use formatted cards and you understand the FAT locations.

One tool that is helpful in understanding and debugging file accesses is a disk editor. One tool I have used is WinHex.
http://www.winhex.com
When I did this several years ago, their demo version would let me view, but not modify the disk clusters. This was sufficient for me, as all I needed to do was check the modifications to the FAT that my microcontroller was making. If their current software does not have a demo mode, you can search the web for older shareware versions of this program. Or you could try a freeware tool like:
http://mh-nexus.de/en/hxd/

Start with an SD card and format it using the PC. Then copy a single, very small text file to the card. Look at the SD card with the disk editor trying to make sense of the FAT and root directory entries. Make some notes of their contents. Now, using a text editor, open this small text file of the SD card and add some text to it. Save the file and again use the disk editor to see the changes to the FAT. Do this several times and you will get the understanding of how it all works.
Now reformat the SD card in the PC. Put the same small text file on it and attempt to modify it with the microcontroller. Then eject the card from the microcontroller and examine the FAT using the disk editor. This will show you what the microcontroller did correctly and where it messed up.

 

[Deepak350]

disk error onsd

Hi banjo thanks for the reply.

I'm now trying what you suggested. I will format the SD in FAT32 format in my PC and try to write a code to read/modify/write onto the SD. I'v been looking for example codes but have been unsuccessful. Please help me if have any code as such. Thanks.

 

[banjo]

code for sd card initiliazation

I am not using any PIC microcontrollers, so I do not have any example code. Perhaps some other members can help there.

I would suggest reading the documentation for the FAT library you are using very closely. When I was working with AVRs for this purpose several years ago, that library only supported FAT16. Make sure your library is supporting FAT32. If you are using long filenames, make sure that is supported too. Usually, these libraries have strict limits. The reason is code size. If they have to test for too many options, the library gets too big to be useful.

Make sure to close the open file on the SD card before your program exits. Leaving the file open could cause problems.
If the SD card is corrupted after writing, break your code into pieces to determine which section is causing the issue.

 

[Deepak350]

copy files in sequential order to sd card

Hi banjo,

I had given up the SD card project the past few days, I'm at it again. I will first tell you my progress. I formatted the card and wrote a 'test.txt' file onto the SD card from my desktop. So far I could detect the card and check if it is FAT formatted successfully. However the problem is that I'm not able to read a file from the SD. Please help me with that. Hope I'v provided enough information.

Best Regards.

 

[doraemon]

msp430 fatfs

Hello!

You can find a lot of documentation on the net which explains how the FAT file
system (16 or 32) are organized.

So apparently you have solved the raw read and write problem, and you are
able to find the MBR, etc... This is the physical layer.

Now here is how the FAT works (I am writing a new version of FAT16 for
MSP430 now).

All what follows is validfor FAT16 because I am in it, but the method is very
similar for FAT32.

First, you have to find the MBR.

1. Read the very first block of the sd card (sector 0). Find the location 1C6 and
read the location of the MBR. In my case, it is F9.

2. Read the block 0xF9.
You will get a bunch of data from which you can read the card characteristics
like bytes per secor, sector per cluster, reserved sectors, etc...

3. Form this data set, calculate the location of the root sector. And you can
also calculate the beginning of the data area.

4. Read the root sector, which is a collection of file records. A file record is a
32-byte record (therefore in a 512-byte sector, you will get at most 16 records).
In your case, one of these records should begin with "test****txt0".
The * represent a space (0x20) and the 0 represents the 0x00 value.
You have also some information about the creation date, modification date,
etc...

5. Once you found your file record, look at the 2 bytes at offset 26 (bytes 26 and
27). They represent the location of the first cluster of your file. Careful, it's
written in little endian. Therefore (supposing your file "test.txt" is the only
one in the root dir), these 2 bytes will be 02 and 00. But it does not mean 0x0200
but 0x0002. Little endian means "little end comes first".
Careful: in the beginning of the card (management areas), all the calculation
units are in sectors. But in the data area, you have to take clusters into account.
One cluster is a bunch of sectors (32 in the case of the 1GB card I am using).

6. Now you know that your file starts at the second cluster. So go past the root
directory, you will therefore be in the data area, and find the cluster # n-2,
therefore 0 in this case. It will be the cluster that follows the root.
Your test.txt will be there.
For instance, if you wrote the file test.txt with a plain text editor (notepad), and
if this file contains "hello", then you should see:

0005C200 48 45 4C 4C 4F 00 00 00 00 00 00 00 00 00 00 00

The left part is the absolute address in the SD card.

7. Last thing:
Before getting a headache, you should first:
- Format your card
- Write a single file named "test.txt"
- Download winhex
- Choose "open disk" and choose your sd card in the menu.
- Then navigate in the card's data until you (more or less) understand
what all that means.

Have fun!

Dora.

Hi banjo,

I had given up the SD card project the past few days, I'm at it again. I will first tell you my progress. I formatted the card and wrote a 'test.txt' file onto the SD card from my desktop. So far I could detect the card and check if it is FAT formatted successfully. However the problem is that I'm not able to read a file from the SD. Please help me with that. Hope I'v provided enough information.

Best Regards.

 

[Deepak350]

how examine partition table of sd card

Hello doraemon,

Thanks a lot for the reply. The step by step explanation was fantastic. Thanks a lot for taking so much time in writing the same. I will run my code and follow the steps you have mentioned and get back to you. Thanks again.

Best Regards.

 

[joshiboshi]

read root sd card

Hello Doraemon,

I am also currently using an msp430 to try to communicate with an sd-card. Essentially, i would like to read from a txt file written in windows, as you have mentioned in previous post.

I have only begun to start with it. I would like to ask if there is any minimal requirement for the MSP430 other than the SPI interface.

I have seen that there is an appliction note from texas instrument for the hardware interface. But the programm code is for MMC card. Is there a lot of difference??And if it is possible, can you provide your code for reference?

Thank a lot.

Josh

 

[doraemon]

sd card pic hardware interface

Hello!

Sorry for the delay, I didn't notice your question.
The application note from Texas Instruments is for MMC but it can be used
for SD. Basically it's the same, but you don't have to pay to claim that you
are providing SD compatible products.

An SD card should work in a slot designed for MMD. An MMD may not work
if intserted in an SD-only slot. However, usually all the drivers are written
to be compatible with MMD, so there is basically no problem.

The only difference I am aware of between MMD and SD is that MMD should
be initialized with a 400 kHz clock, and SD can be initialized at higher rates.
Coclusion: just in case, set a 400 kHz clock for initialization so that it works
for every type, and then switch to a higher rate.

As for the footprint for MSP430, I would say that 2 kBytes RAM is reasonable.
I think it could work with less. My implementation uses about 800 bytes RAM.
However, if you take for instance a 1K device, be aware that you should
leave some space for the stack, so although I would say you can do it
within 1K, you have to be aware that there is not very much space remaining.

I did some development in the past on PassWorld's development board with an
MSP430F169. It has 2K RAM, and it worked fine. **broken link removed**.

But you may consider using newer devices (the new F5438 is cheaper, and it
has 10 K of RAM, works at 25 MHz, has an incredible math engine that I used
for 256 point FFT, etc...).

The code I have developed was for a contractual work and therefore I cannot
pubished it. However, it was based on this document.
For your information, this code works as is. Copy and paste it into files,
compile and link them together and that's it.
The code is slow, but it works.

Dora.

Hello Doraemon,

I am also currently using an msp430 to try to communicate with an sd-card. Essentially, i would like to read from a txt file written in windows, as you have mentioned in previous post.

I have only begun to start with it. I would like to ask if there is any minimal requirement for the MSP430 other than the SPI interface.

I have seen that there is an appliction note from texas instrument for the hardware interface. But the programm code is for MMC card. Is there a lot of difference??And if it is possible, can you provide your code for reference?

Thank a lot.

Josh

 

[sri1234]

sd card txt file create

hi to all,
i;m using sd card with PIC 24 series, i have already go through the forum i have tried a lot but card s not sending response(0x00) to me for the com0, for com0 i'm sending (0x40,0x00,0x00,0x00,0x00,0x95);

pls support me in this reg,

 

[chethankp]

Re: sd card txt file create

Hi,
Im working on SD card read/write using MSP430F1611 and the file system is FAT16.

I just i'm trying write few characters to the file on sd card. Before writing in to the SDcard I formatted the SD card using EXTRENAL card reader then I put it back to my circuite and ran my code, after this I opened the sd card from card reader ,its created the 32kb file and written few character(which want to write ) and its dumped some dummy data to the file, dumped data was from the file which I had Formated first time.
what is probelm,how its taking the data from the file which is not there and dumping to the new file..?
How to initialize file with empty ..?
regards
chethan

 

[doraemon]

Hello!

It's not possible to identify the problem without further information.
It's somewhat like if you say "my car makes a strange noise, what happens?".
What you may consider to do*
- Format your card
- Try to write one file with a few characters only. For instance as I wrote earlier
"hello" which translates in 48 45 4C 4C 4F 00 in hexa.
- Verify in the FAT records that you have a file named for instance "hello.txt"
(it will be hello___txt in the record, _ being a 0x00).
- Verify in the FAT record that the first sector is 2 (again, I am talking of a fresh
formatted card).
- Calculate the position of the first cluster of your file. The first relative cluster
is 2. You should calculate the absolute cluster.
- Calculate the offset in byte of this cluster
- Move to this absolute offset, and check if you see 48 45 4C 4C 4F 00, the contents
of the file.
- I guess that's about it.
Note that after the 00, you may have garbage remaining from a previous use. That's
normal. When you erase the file, you erase only its entry in the tables, not the data.
It's like if you remove a title from a table of contents. If you look by the table of
contents, you will not find what you are looking for, but if you browse the complete
data, you will.

That's about it!



Dora.