Some of the best stores for running gear are local chains, so your choices may largely depend on where you live. Here in Los Angeles, A Snail's Pace is one of the best for running shoes. They're a small but very well-stocked chain (4 stores total).  For running clothes and socks, I like the Sports Authority (which is nationwide) and also Sport Chalet (mostly CA).

For shoes, I either go to the local specialty running store or buy online directly from the manufacturer.  Although, I don't suggest the online shopping for "newbies".  You should only go the online route once you've been properly fitted and know what shoe you should be running in.  For everything else, I buy it as cheaply as possible.  I mostly shop off the clearance rack at the national chains.

I shop locally when possible but am loathe to pay full price for anything. My local sporting goods stores are good, and I also have found a surprising number of things (track pants, tech shirts, shorts, etc.) at markdown stores like Ross and T.J. Maxx. They don't necessarily have "this year's style" on the racks but the prices are right.  I buy things online too if I find a great deal... Recent online purchases were from Lands' End (nice Thermaskin long-sleeve shirts and capri tights in the Overstocks area), the Adidas Outlet (trail running shoes), Amazon.com, and eBay.  I figure it helps keep our local UPS and FedEx drivers employed too. 

One of my favorite sources for gear and shoes is the trade fair / expo at races, but I live in Montana where our races and associated trade shows are small. But I travel to the Lilac Bloomsday 12k race in Spokane, WA every year, which is huge. Last year I found my all-time favorite cool-weather running top (SportHill brand) on sale at their expo. Plus, you can try on shoes and get advice from all the different vendors. Super fun! Can't wait until next year's race!

Okay one thing to keep in mind do not buy running shoes from ebay.  You do not know how long they have been laying around and they will break down with age.  Even if you're buying your favorite brand and you've been fittted for them.  They loose their integrity with age, even if they haven't logged one mile.