The paper describes design and implementation of software infrastructure for building augmented reality applications for ubiquitous computing environments. Augmented reality is one of the most important techniques to achieve the vision of ubiquitous computing. Traditional toolkits for augmented reality provide the high level abstraction that makes it easy to build augmented reality applications. However, the applications programmers need to contemplate distribution and context-awareness that make the development of applications very hard, but they are necessary to build ubiquitous computing environments. Our infrastructure provides the high level abstraction and hides distribution and context-awareness from programmers. Therefore, the cost to develop augmented reality applications will be reduced dramatically by using our middleware infrastructure.