I have used a Dell Axim Pocket PC for years for my paperless geocaching. You will need to load an application on it for the paperless caching. I use GPXSonar, which was free. There used to be a link to the download site for this from Geocaching.com, under the Resources section. However, i see that the link is no longer there. I checked the GC.com Forums and found this discussion, which says that GPXSonar is no longer supported, but still available. Also, I'm not sure whether it would work with your version of Windows on your Pocket PC. But it's worth a try. Others might suggest another application which is still supported.
GPXSonar works great. I use GSAK to interface with it. I first run a pocket query on GC.com (you have to be a premium member to use this feature) and then open the zipped file into GSAK. I then save the information into a gpx file which I copy to the Pocket PC and open with GPX Sonar.
If you don't have a handheld GPS receiver unit, you need to get one! Put that at the top of your "to get" list if you are serious about geocaching.
"Wildness is a necessity." -- John Muir
"When you go to hide a geocache, think of the reason you are bringing people to that spot. If the only reason is for the geocache, then find a better spot." – briansnat