On a generic Windows box (95/98, NT, 2k, whatever):
1. Make the swapfile minimum and maximum size the same. This keeps the OS from the expensive operation of resizing the swap file, and keeps the swap file from getting fragmented and introducing more fragmentation. In the 'System' Control Panel-->Performance Tab, click the 'Virtual Memory' button (win9x) or the 'Change' button (NT). From there, you can specify the minimum and maximum to be the same size. I would recommend a size of physical RAM plus 11 meg (139 swap for 128 meg of RAM, etc).
2. If you have two physical hard drives, I would recommend putting your program/OS binaries on one drives and your swap file and other static data (pr0n, mp3s, etc) on the other. If these are IDE drives, put one drive on each IDE "channel" (bus); if they're SCSI, you'll get concurrency between the drives anyways. You can specify the swapfile location in the same place as in step #1: Control Panel-->System-->Performance Tab-->Virtual Memory (win9x) or Change (NT). On NT, the UI is a bit confusing -- to remove swapfile presence from a partition, enter zero for both the Initial Size and Maximum Size and click 'Set'. When you completely remove the swapfile from the partition the OS is on, it may complain. It is alright to leave the 2 meg minimum on the OS partition so it can give crash dump infoz.
3. Install the latest updates from Microsoft. On NT, first install the latest service pack (http://www.microsoft.com/ntserver/nts/downloads/recommended/SP6). On all platforms, install IE 5.01 (http://www.microsoft.com/ie). When that is finished, run IE and select Tools-->Windows Update. Download all the critical updates and the Virtual Machine updates, which may take a few rounds to complete.
4. Install the basic set of utilities: Winzip (http://www.winzip.com); Adobe Acrobat (http://www.adobe.com); unzip contig, pagedefrag, and strings into your PATH somewhere (http://www.sysinternals.com); and optionally Cygwin (http://sourceware.cygnus.com/cygwin).
5. Then, go to the web sites for your various pieces of hardware and make sure you are running the latest drivers availible. (You may also want to see if there is a newer BIOS availible from your motherboard manufacturer and flash your BIOS.)
6. Turn off all the unnecessary animations and sounds. In the Control Panel-->Display-->Effects/Plus tab, it's alright to uncheck all the boxes. I prefer to leave 'Show window contents while dragging' on, though. In the Control Panel-->Sounds window, set the Scheme to be 'No Sounds'; do not save the previous one when it asks.
7. In the Network Control Panel, remove unnecessary protocols/services. The one that causes alot of slowdown is IPX/SPX, which you don't need unless you are planning on accessing Netware file/print servers.
8. Remove unnecessary file clutter from the OS install. Remove *.txt, *.log, and *.bmp from your OS directory (C:\WINDOWS, D:\WINNT, whatever) and your root directory (C:\). Win98 leaves a lot of *.htt and desktop.ini files around, do a find for them and get rid of them all. Clear out your TEMP directory (C:\WINDOWS\TEMP on win9x, C:\TEMP on NT, and the TEMP directory under your user profile in Win2k). On Win9x, totally remove all the Online Services crap from your Desktop, Start Menu, and C:\Program Files directory.
9. Defragment your drives. First, clear out your browser cache and history. Then do the following:
On Win9x, run sysedit. In the SYSTEM.INI file, change the line that says "shell=explorer.exe" to "shell=progman.exe". If the "shell=" line is not present, add it under the [boot] section. Reboot into DOS mode (Start-->Shutdown-->Restart in DOS mode) and run "scanreg /fix". This will compact the Registry database to be smaller in most cases. Reboot into Windows and you should see Program Manager instead of regular Windows Explorer. From File-->Run, run defrag and let it do a complete defragmentation. When it's done, run "sysedit" again and change "shell=progman.exe" back to "shell=explorer.exe".
On NT, you'll need to first compact the registry; information on this process can be found @ http://support.microsoft.com/support/kb/articles/Q247/9/20.ASP . Then, run the pagedefrag utility mentioned in step #4. Set it to defrag the registry and paging files on next boot. Reboot. Next, bring up the Task Manager (Control+Shift+Escape) and end the Explorer.exe process. Then, from task manager, File-->Run cmd.exe. From the command prompt, go into your WINNT directory and delete *.txt. then, change back into the root directory. Run "contig -s *.*". Let it go until it's finished. Then run pagedefrag again and have it defragment the registry and paging files on next boot.
On 2k, just kill the EXPLORER.EXE process and stop all unnecessary services. Then File-->Run cmd.exe from the Task Manager, just like in NT. Delete *.txt from your WIn2k directory, and then close the command prompt. Run File-->Run defrag and let it run until it completes. I find it's usually good to run defrag twice under win2k, as it's using the diskkeeper backend which is fairly retarded.
Some additional notes: Avoid installing Netscape if possible, it copies over your known good system DLLs which very old/buggy versions. On Win2k, with System File Protection, this scenario is prevented. On NT/2k, use NTFS; on win9x, use FAT32 if it's availible.