In a medium saucepan, on medium heat, bring the water to a boil. Add the star anise, cardamom, cloves, and cinnamon stick.
Cut the vanilla bean lengthwise and remove the seeds. Add the seeds and the vanilla bean to the water.
Remove the pan from the heat and add the tea bags. Allow the tea to steep and cool to room temperature, about 1 hour.
Remove the spices and the tea bags. Whisk in the condensed milk.
Pour into a large pitcher and refrigerate for at least 1 hour to chill.