One of the many new features of the iOS 4 is the improved data encrypted file system (which is enabled by default on an iPhone 4). If you upgraded your iPhone 3GS or iPod touch (3rd generation) to iOS 4 the feature known as 'Data Protection' will not be enable. To enable this functionality on your device, Check out the following article from Apple for the instructions: iOS 4: Understanding data protection.
Note: This is a long process that will erase all user content from the device, then restore it again. Proceed at your own risk.