Answers

I would recommend starting with making sure all the existing windows and securely closed and locked.  I find windows partly open because they have been painted that way.  Take some time to make sure each window closes properly add weather stripping as needed.

Add wather stripping to all doors

check that all the heating ducts are connected securely

The best thing you could do for your home to keep the warm in and the cold out is to 1st. check your insulation in your attic if your not properly insulated the heat will escape. Another is making sure your windows are secured and latched. and proper weather seal is on your doors so draft cannot come in. 

I'm from Wisconsin and we have a program called Focus on Energy which implements the ENERGY STAR program for improving energy efficiency of older homes. I'm not sure what you have in your region, but I specifically recommend starting with an expert company that can do a blower door test on your home and use an infrared camera to detect where you have air leakage and heat loss.

The number one cause of heat loss is air leakage. So insulation alone will not solve that problem. Leaky ring joists in the basement where the walls, floor, and foundation meet are one culprit... there is generally lots of inward air leakage here. And in the attic, there are a bunch of sources of air leakage, where warm air wants to rise and escape up and out. (So by the way, ice dams on the roof are not solved by adding more attic ventilation; rather they are solved by first doing air sealing, and second verifying or improving insulation.)

If you intend to DIY this, you can still hire a consultant to do the pre-testing and post-testing, and you might even be eligible for some financial incentives. If you hire a professional company to do it, the cost can be reduced by those incentives.

If you won't hire a pro, then here's a few rules of thumb: 

1) Remove fiberglass insulation from ring joists, and either use spray foam or rigid foam to insuate the ring joist, use spray foam to seal the rigid foam in place, minimum 2" thick and you can always fit the fiberglass insulation back in place again when complete.

2) Spray foam over top of wall plates in the attic.

3) Put a gasked on your attic hatch. If you have an attic ladder, buy a specific air sealing enclosure to prevent air leakage through it.

4) Find out if your recessed can lights are IC (Insulation Contact) rated or not. They will be labeled if they are. Build a sealed box around them allowing air space for heat build-up, and consider converting to LED lights so that there is less heat generated. If not IC rated, use cault to seal them to the drywall or plaster, and to close up the holes in the lights themselves.

That's a primer on things... there is more to be done, but these can help!

Energy efficent windows are a good investment.