I'll try to outline a few important things that everyone should now when making a decision about buying a new motorcycle. The general rules are quite similar to those found when buying a typical street bike. Do not overestimate your skills as more power is not necessarily better. It's not very wise to start with a very strong bike just because 500 sounds better then 250. Horsepower is not that important in off road riding and you won't reach a point when “it's not enough” as soon as you could in street riding. Let's focus more on different motorcycle specifics that are important.
Two-stroke or four-stroke?

To answer this question I'll try to highlight a few details of this two types of engines. A two-stroke engine doesn't have a valvetrain, so it's much more simple than four-stroke and at the same time capable of generating more power from lower engine displacement values. And a smaller engine means a lighter bike overall. Yet, two-stroke engines have some serious drawbacks. They're rather difficult to control as the power is generated in a very aggressive manner (similar to supersport street 600cc motorcycles).
At low rpm the engine is steady and calm and then, all of a sudden a giant power unleashes. This is much easier to control on street as the back wheel has a good tracktion. In off road you have to be much more careful, a slight move of your right hand, opening the throttle just a bit too much and you're out of control. 250cc two-stroke motorcycles generate around 60 horsepower, and this compared to their overall small weight (often less then 100kg) makes a bike really difficult to control. Conclusion is simple: beginers should avoid too strong motorcycles.
On the other hand four-stroke engines are considered more durable. This is partly truth, but professional four-stroke units usually require a lot of service job and this on a long run can make them similar to two-strokes in financial terms. Power and torque characteristics are also different, it's generally much more predictable. Smooth gas dosage makes the bike easier to feel and lets the biker progress his skills faster.
Personally I believe that two-strokes are better for track riding and four-strokes are better for natural (and unknown) environment. On the track you usually know every turn and every slope, you can plan ahead and so the engine won't surprise you as you should always be prepared for every maneuver. Four-stroke on the other hand is perfect for unknown places. It makes it easier to handle slopes and accelerate smoothly on every gear.
So what motorcycle should one choose?
Let's assume that we have a specific amount of money to spend on a motorcycle. Buying the motorcycle is only the first part and the second one (and very often more important) is supporting it. Off road motorcycling is usually much more expensive then street motorcycling. Why is that? The reason is supporting and servicing your motorcycle. And what exactly is so expensive? Oils, chains, tyres, brakes, bearings, suspension parts and all kinds of mechanical breakages. In spite of low mileages of off road riding you have to do a full bike service many times during each season. Remember: jumping, sudden braking, much gear changing – everything you will have to do in off road has a negative impact on your bike and on your wallet.

If you have infinite money to spend then you could just buy a new bike straight from a shop. Just don't forget that it can look like a wreckage after only one year. This won't save you the support/service expenses, but still any serious failures should be handled on warranty terms. It is worth noting that new four-stroke engines are generally a good investment, because of a good durability and a very long mean time before first failure. So if you've got the money – buy a new four-stroke bike.
What you should look at when considering a used bike? The important factors are: total age and engine condition. Other important things are weight and size. Remember that every single kilogram in off road counts twice. It's important to match the motorcycle to your physical attributes. If you're two meters high and weight over 100kg then you shouldn't have any problems with lifting a 130kg bike. But the problem may come when you'll have to do it in deep water or mud. It's a good idea to check a few different models, see how you feel with them, before choosing the final one.
Another important factor is seat height. It's usually more than 90cm and if that makes you feel uncomfortable then you should check if your chosen model has any lowering kits available.
You should also know where are you going to ride your motorcycle. In extremely difficult terrain it's quite nice to have an electrical starter except the standard kick starter. Can you image trying to start your bike on an extremely bold slope using a kick starter? On the other hand kick starter can be a life saver if your bike battery dies. The only serious drawback of having an electrical starter is the weight increase (battery and the starter mechanisms are quite heavy).