Dataloggers, display and music players all need to store a lot of data, way more than the internal EEPROM of a microncontroller. You might also need to easily move these big files between a microcontroller project and a computer.
The best way to do this is to use camera/MP3 player flash cards like SD or microSD.

http://www.ladyada.net/wiki/partfinder/connectors#memory_card_connectors

The larger SD card holder is not a push-to-eject type but its by far the easiest we’ve found for soldering. The smaller microSD slot is push-to-eject type but tougher to solder. We have EagleCAD footprints for both in our Eagle library. And we have a handy microSD card breakout board that you can use with any microcontroller