The duct work in your attic should be totally sealed. If it's not, especially on the "return" side (air being brought back to be conditioned) air is being sucked in from a very hostile environment. This air can be 140 degrees in the summer and less than 10 degrees in the winter. An air conditioner is only designed to lower the temperature of the air by 20 - 25 degrees. If this attic air mixes with the air from the house before it reaches the equipment it could raise the temperature by 30 degrees, depending on how much air is being sucked in. Imagine trying to cool your home with 80 degree air! Think of all the extra electricity it takes to heat air from 10 degrees to 70 in the winter, too.
This filter access is the spot where the duct work is most negative, meaning it's pulling the hardest. Can you see how air is being dragged into the ducts through a hole this size?