That is my whole collection of gemstones, minus the ones I have sold or set in jewelry. I have been collecting gemstones for about 8 years. I started because I just loved them, there are so many. Oh this doesn't include the cabachons I have either. As I started to really learn about gemstones, I really wanted to collect rare and exotic ones. The rarer the stone, the more expensive. Always by the largest and best quality stone you can afford. If that means you can afford a 4mm round Blue Diamond with good color and clarity, then that is what you should buy. Then work from there.
I mostly buy in large parcels, then have to try to figure out what they are. You usually can pick out the unique or better quality stones from just general stones. A lot of parcels will contain Amethyst, Peridot, Garnet, Citrine and Blue Topaz. They are the most abundant and least expensive stones on the market, for the most part. These are great for starter collections and making jewelry. Most people know what they are, and would be willing to buy them. There are some from each of those families that are more expensive, and of course bigger carat sizes draw a bigger price tag. For example Tsavorite and Mandarin Garnet are more valuable than traditional red Garnet. Imperial Topaz is extremely rare and expensive. Natural Swiss Blue Topaz is very difficult to find, and Madeira Citrine is a natural form of Citrine that looks like red wine. These are all highly collectible gemstones from their traditional counterparts.
So what I am saying is, buy the large parcels, you will get some special stones, but for truly collectible stones search for the exact item you want. Know what a quality stone looks like, and sells for on the retail market. Then go hunting for a deal. They are out there you just have to look and be patient. It took me three years to find an Alexandrite that not only could I afford, but had good quality. I love Alexandrite, and all color change gemstones.
I buy loose gemstones from JTV, Rio Grande, Fire Mountain Gems, and Ebay. I find Ebay you can get a good deal by buying overseas. The shipping is higher but they sell their gems for a fraction of what they do here. Just be sure to read their feedback and ask all questions before you buy. An honest seller will answer all questions. I have looked at other websites, but I find their prices to be exceptionally high. Like most people, I love a good deal. Read, read, read all the information you can before you begin. Educate yourself and be knowledgeable.
Know what flaws a gemstone may have and how it can affect the price. Because I love the more rare gemstones, I don't mind a flaw here or there. Also ask about treatments done to the gemstone. All sellers should know this info, it is required to be disclosed by law. If they don't know and the gemstone is really really high priced, I wouldn't buy it. If the price is decent I might take a chance. Resellers might not know, especially when they buy large parcels like me. If you don't have a way to test for treatments, you won't know. Most gemstones are treated, I think something like 95% of them. Be cautious of words like all natural. This usually means the gemstone has no treatments to it what so ever, but this is not usually the case. Sometimes all natural is used to mean it is a real gemstone not a fake or faux. So ask questions, always.
I think the last and most important thing is, don't buy if they don't offer a money back guarantee, unless of course it is the best price you have ever seen and just can't bare to not have it. If they aren't willing to take it back, that is usually a red flag at least for me.
Please note I am not an expert, just an enthusiast and learner of gemstone identification. Always know your facts before you make a purchase from anyone.