I'm in the process of listing all the Big O merchandise that I know of. When possible, I also give links to Amazon.com, which is where I buy most of my stuff. I've also heard a lot of good things about Anime Nation, which has a very complete line of Big O stuff; much more complete than any of the general-interest stores.
Big O Anime DVDs
You should run right out and buy the DVDs. Really! They're great! Links to Amazon.com, are given below. If you see a generic ad, they're probably out of stock on one of the items. Try one of the other online sellers (like Anime Nation or one of the ever-changing Google ads at the top of the page).
I've listed the boxed set for Season 1, the new boxed set of Season 2, and the individual Season 2 CDs.
Big O Soundtrack CDs (OST)
While I'm listing things, I might as well list the soundtrack CD's. I love and recommend the first one. I don't have the second one yet.
Big O Manga in English
The manga are published by Viz. They're quite a bit different from the show. Some people like them a lot.
Big O Art Book
I love the art book, even though I can't read the Japanese text. Most of the headings are in English, though, and the art is superb.
Big O Toys and Action FiguresThere are some cool Big O Figurines.