Believe it or not, a bigger tank is actually easier for a beginner to manage. Small tanks tend to be prone to temperature swings and can require more frequent cleaning. A 10 gallon tank is a nice, happy medium if you're wanting to start small; I loved my 10g setup to pieces, and it was easy to maintain even while in a college dorm.
A tank is easier to manage than a bowl because you don't have to change the water as frequently to keep your fish healthy, and you are less likely to cause the fish harm by changing the chemistry and temperature of the water suddenly between changes. In a tank, you can change part of the water instead of doing all of it at once. You also have a filter and a bacteria community which will help keep your tank cycled and ammonia free. If you don't know about the nitrifying cycle, read about it before getting any fish; they will be much healthier if they are placed in a tank that is pre-cycled, if possible, or a tank that is safely cycled while they are in it.
Goldfish are related to koi, and can grow to be too large for smaller tanks. The common comet goldfish can be 8 inches long. They are also huge waste producers, and their tank will need frequent water changes and vacuuming.
Avoid livebearing fish such as mollies and guppies; they breed quickly and can easily overwhelm your aquarium.
Start with sturdy fish, such as danios and corydoras. Unfortunately, many of the "pretty" fish (such as gourami) are also delicate.
I'm fond of bettas; they're great little fish, and easy enough to manage if you find one that isn't sick to start with. The problem is finding a healthy betta at the store, because most stores keep them in cups where the ammonia levels skyrocket and damage their gills. If you want a betta, ask the store what days they restock the bettas and go on that day to pick out one that hasn't been there very long (in fact, this is a good way to do it for ANY kind of fish you get, since store tanks are prone to disease). Be incredibly wary any store you go to, especially chain stores such as WalMart and even Petsmart and Petco. Inspect their tanks closely for signs of disease and dead fish. Also look for an active fish who is moving around a little with all his or her fins spread. Some may be lunging at their neighbors, some may just be resting peacefully. I go for the ones who aren't aggressive, because I keep a community tank, but if he or she will be your only fish, pick one you like.
Research, research, research! Read about any fish you consider adding to your tank until you think you know everything there is to know. What temperature does the fish live in? Do the other fish in your setup live at that temperature? Will you need a heater? Does the fish need a school (like tetras), or are they happy alone (like bettas)?
It might be a good idea to go in to the store and just look around and see what you like, write down the species names, and then go home and research what will work best for you.

This forum is one I've found to be a wonderful resource: Aquaria Central - the folks there are generally happy to help and very knowledgeable.

Make sure you understand a fish's personality. There is information about whether they are friendly toward other fish or whether they can be aggressive or even lethal. Betas for example can attack other fish, which is often why you only see them swimming alone.

You can actually buy a small bowl or tank for a beta and just have one of them, it just may not be the kind of thing you would want, but we've owned a small tank before just for a beta. Big tanks are nice but I will warn you that cleaning them was always a hassle for me.

Probably gonna be repeating some of what Aivilo said, rofl.

I have two bettas and admit that I am a beginner, but I did research. If you get a fish, regardless of what type it is, make sure it has space. Don't coup it up in a tiny bowl that's a gallon or less, or they won't live for very long, or be very happy. Both of mine have a 5 gallon (a 5 gallon is actually the minimum amount of space any single betta should have) of their own, with a gentle filter and a heater. Even with the filter, they both need about a 15-20% water change a week to keep things stable, and they need water conditioner too. A gravel vacuum/cleaner is also a big help.

While bettas do have their needs and are very sensitive and delicate fish, I find their individuality quite rewarding, because never have I had fish that spazz upon the very sight of seeing me, or feed from my fingers, and do all these other sorts of cute things. All the money I spent for my Maurice and Jelly was worth it. c:

Oh ohhh, about decor and bettas (if you get one in the end); Make sure anything you put in there is not sharp or has a hard texture. A betta's fins are about as delicate as panty hoes. If you run a pair of panty hoes over the object and it snags even the slightest bit, your betta's fins run the risk of being torn. They're very delicate.

When you clean a tank by hand, instead of using soap or any kind of chemical cleaner, try sea salt! Once cleaning time rolls around for my bettas' tanks, I rinse it out (first after taking the fish out into a coffee cup filled with already treated water!) with tap water, then leave a tiny tiny bit in the bottom, and dump about half a tablespoon of sea salt into it. Then just grab a paper towel and scrub the salt everywhere for a few minutes. After that, I rinse the tank out with water that I've allowed to sit and evaporate for a few days (where I live, they treat the water with fluoride, which is good for humans but bad for fishies! instead of buying "fish water" that has nothing added to it, I have one of those glass jugs you can make sun tea in and I leave the top off so the chemicals evaporate out over a few days). I rinse it again just to make sure all the salt is gone, put the fish's stuff back in (I scrub that with salt, too) and all done!
Cleaning with salt sounds weird, but we had this one betta that lived about 6 or 7 years and that's all we did for his tank.

Also when you pick out a uhhh... bottom material for your tank (like pebbles or marbles) check it to see if the material has been painted. Sometimes if you get really cheaply painted marbles (like my mom did) the paint will flake off and one day your fishy will think "oh, a snack!" and eat it, which is no good.

^Yes!
The exception being 4+ females, but even then you've got to know what you're doing.

And the word you were looking for is "substrate," Biscuit xD
I prefer large white+brown or all black gravel substrate, personally, just for the way it looks, but you can find all kinds of colors and sizes.

"-Y'all lost me on the different types of water for the fish. I'd thought tap water would be enough."

Tap water contains chlorine in small amounts in order to keep bacteria levels down and make it safe for human consumption. That small amount is highly toxic to fish, however, as it damages their gills. Fish require water which has been treated with a dechlorinator.
Sometimes cities also add other chemicals, or the water source itself has other substances, which are also toxic for fish, and those must be removed as well.

All that aside, you really only have two basic types of water: freshwater and saltwater. You can make it more complex with that, especially with saltwater, but those are the categories most fish fit into.

A 5 gallon aquarium is reasonably small. A standard boxy one is 16" x 8" x 10", but you can also find hexagon shapes that might take up a little less room.

A heater can help balance fluctuating room temperatures, although it gets harder in the summer than it is in the winter.

As for the rest, yeah, it takes a lot of effort to learn/remember a maintainence schedule and how to differentiate healthy fish from sick fish (you have to see a lot to know what you're looking for), and funds are generally hard to get around if you just don't have them.

It may not be something you're able to do now, but if you're really interested and in a few months or years you want to go for it, at least you have a starting point now :3

Just gunna slip a track in here since this is all pretty helpful.