Answers

Gaps are typically caused by different expansion and contraction rates of different types of materials at different temperatures. The frame of the window may be vinyl or wood, which expands at different rates than the wood wall framing and the brick exterior wall surfaces, depending on the temperature or humidity of any particular day. Some days (or seasons) the gaps will be larger and other times they will be smaller. Using a window frame made of fiberglass, which has very low expansion and contraction rates, would be a good solution for this type pf problem.

 The quality of the caulk used to fill those gaps will also determine if it will stay attached to those surfaces, and the caulk will also expand at different rates. Generally, the thicker the caulk bead, the more you will see these negative effects. Thicker caulk beads, typically used to seal larger gaps, have a harder time staying attached to both surface they are sealing up and will come apart more often. Therefore, it is important that the window is sized properly and installed well in order to avoid large gaps around the window that then need to be filled with caulk.

I agree wholeheartedly with the excellent answer provided by Christi at Callen Construction that your problem is probably caused by the different coefficient of thermal expansion of the materials involved.  And yes a fiberglass window would be a good solution, but a wood window would also solve the problem.

To put some numbers to the expansion and contraction properties of window material, with 10 as the worst performance and 1 as the best, here's how they stack up:  vinyl - 10, composite (like Andersen Renewal Fibrex) - 4.5, actual fiberglass (like Pella Impervia or Marvin Integrity) 1.4, wood 1.0.

Don't discount a clad wood window if you like the aesthetics- it will perform well if properly sized and installed.

Expansion and contraction, as mentioned previously, is a big driver of sealant failure.

The other part that was not discussed in moisture content.  Brick is largely moisture open and capillary draw of moisture will keep the brick wet and can cause the sealant to fail. 

If the home is new construction, the sealant that was used is cheap in most cases and was probably poorly applied.  Sealing large gaps requires the application of backer rod or some sort of backstop to apply the sealant to. 

The expansion rates are different for different materials but they aren't as dramatically different from one another often thought of. 

For example, Fiberglass is often touted as a much better material because of its more analogous coefficient of linear expansion with glass than vinyl.  That is true but it is not 7X less likely to expand than PVC.  More like 2X in most cases. 

Wood has a verly low expansion rate as related specifically to temperature but is moisture driven as Christi was referencing with the humidity statement. 

It seems like the caulk must have come loose. My recommendation is to recaulk the area and make sure that the gap is filled with insulation. Another option to look into is double expanding foam. Hope this helps!