Have you done it? What good markets are there? Are the risks higher than for 1st world markets? Are the returns better due to the higher risks involved? What sort of companies have you invested in? Do you expect them to grow significantly over the next decade?
I do not directly own any shares from 3rd world countries. Information can be hard to find and liquidity can become a problem. That being said, I wouldn't be against owning a mutual fund with some exposure.
Personally, I would stay away for now. crimson is right that there is little information or transparency. Also, I would worry about corruption. Many "third world" countries are in the position they are in because of government corruption and I wouldn't feel comfortable investing funds in such countries at all.
You're better off investing through a third party (AKA a company with exposure in third world investments) rather than investing yourself. You're only asking for trouble if you take your money to such countries. There are plenty of investments to be had, and a lot of them are going to take you down a path that you won't want to be on a few months or years down the road.
It really depends on how you define "3rd world". There are stocks in more evolved developing markets, such as South Korea, which have decent liquidity, transparency and corporate governance. But if you talking about some of the frontier markets like Vietnam or Nigeria then really you may have issues with all of those, in addition to it being prohibitively expensive to get into the market. If you can get a decently priced one, look for exposure via an ETF or certain funds.
^ This. I have invested in Emerging Markets funds and have, more or less, done fine with them. If one means unstable parts of the world, forget it. Someone may be able to make money in such an environment, but that is too much risk for me.
There are some 3rd world countries which I've heard are relatively safe to invest in. But since most of these countries always have issues at times when they are electing new leaders it easy for you to lose your money should the company you invested in incur losses [at that time] which will take a long time to recoup. Such instability makes investing in 3rd world countries a high-risk venture.
And to add another point of view into this conversation: Currency risk. Remember that most 3rd world countries have a very volatile currency so it can have a dramatic impact on your investments value in the long run.
I'm with most people above in terms of risk, transparency, and just general world markets as a whole. While I'd be worried about corruption as well, I think that's present in the western world already in lesser, perhaps even greater degrees. All that in mind, I've never invested in that space myself and the amount of due diligence I would have to do to be comfortable with it would just be enormous.
Emerging markets have the potential for high gain, but it is coupled with extremely high risk. A major problem is that the accuracy of financial data is questionable. I would be highly suspcious of companies that are majority owned by locals from the region. They can easily manipulate stock prices and legal action against them is very difficult if not impossible.