The problem#
So you've built a new computer with fresh blank storage. How do you actually do anything with that computer that has no software? Navigating the BIOS menus can only hold your interest for so long.
The solution#
The old way of doing things was to have a bootable CD or DVD, but now that most computers don't even have an optical drive, the common way to handle this with bootable USB flash drives.
Most Linux distributions' default download is an image for a bootable "live" flash drive (or DVD) that runs the OS in addition to having an option to perform a permanent install. Some of the most popular ones are Debian, Ubuntu, Mint, and Fedora.
You can boot Windows off a flash drive using Hiren's BootCD, which also includes a lot of recovery and diagnostic tools.
While most boot drives will boot into Linux or Windows, there's a small set of specialized lower-level tools. One very useful one is Memtest86+ (included in many Linux distros), which will determine if your RAM is functional. As bad RAM can cause very weird and different to track down problems, you should always test new RAM.