Remember that pop tart that you turned on its side? Go ahead and stand it up again. Place it next to one of the house-shaped pop tarts, lining up the corners, and cut horizontally across. Repeat.
You should now have two roof pieces (un-cut), 2 house-shaped pieces, and 2 wall pieces.
Combine powdered sugar and water to make a royal icing glue. Add water 1/2 teaspoon at a time until desired consistency is achieved (2 1/2 teaspoons works well for me). Spoon icing into a Ziploc bag and cut the tip off. Glue the 4 walls together with icing and hold in place for a minute or so.
Add the roof pieces and hold in place for a minute or so.
Let houses stand AT LEAST 30 minutes before decorating. If making these with kids, assemble the houses the day before you want to decorate them to allow the pop tarts to harden a bit.
Dip decorations into icing or pipe icing onto houses to decorate.