Augmented Reality (Fall 2022)

Taught by Dr. Zhao Han, Augmented Reality is a Fall 2022 course (CSCI 498A/598B) offered by the Department of Computer Science at Colorado School of Mines.

---

Course Schedule ↓    Canvas Course Page ↗

Course Details

Course Overview

Course Code:	CSCI 498A/598B
Title:	Augmented Reality
Semester Year:	08/23/2022 – 12/15/2022
Credit Hours:	3
Class Meeting Times:	Tuesdays, Thursdays 3:30pm – 4:45pm
Class Location:	Anderson Hall 330

---

Course Staff

Instructor

Instructor:	Dr. Zhao Han (He/Him)
Instructor Email:	zhaohan@mines.edu*
Instructor Office Location:	Brown Hall 280M
Instructor Office Hours:	Tuesdays, Thursdays 10:00am – 11:00am or by appointment
Instructor Website:	zhaohanphd.com

Teaching Assistant (TA)

Teaching Assistant (TA):	Abdulelah Talea
TA Email:	atalea@mines.edu*
TA Office Location:	CTLM 258 (subject to change)
TA Office Hours:	Mondays, 1:00-2:00pm or by appointment

* Feel free to contact the instructor and TA via the discussion forum on Canvas for questions or via email or other communication. Expectations of communication can be found below.

Pre-requisites

• Software engineering (CSCI 306)
• Linear algebra (MATH 332)

Required Technology

I recommend using your own laptop or desktop with a webcam to augment reality. It must run Windows or macOS because we will install and use Unity and Vuforia, and Vuforia does not support Linux.

Canvas Course Website

As part of the learning experience at the Colorado School of Mines, our class will be utilizing online learning resources and experiences through the Canvas learning management system: elearning.mines.edu/courses/42097 (When not logged in, this URL will direct you to the e-learning homepage).

In order to help build an inclusive community in this online learning environment, you are encouraged to

• upload your profile picture to Canvas,
  • Photos should be similar to the photos taken for passports or state identification cards.
• set your pronouns (why?),
• and add a preferred name.

Welcome to Augmented Reality

The field of augmented reality (AR) is rapidly developing both in academia (with two conferences in the top 20 venues of human-computer interaction) and industry (e.g., related to Facebook’s Metaverse plan). I hope you will enjoy learning different aspects of AR and developing AR apps in this course!

Course Description

This course provides a comprehensive overview of augmented reality (AR) concepts and techniques and teaches how to develop AR applications, i.e., creating and blending virtual content with the physical world.

Topics include use cases, hardware, technical foundations such as tracking algorithms and coherent rendering techniques, situated visualizations and interaction techniques, evaluation, as well social aspects.

Programming assignments, short homework exercises, and team-based projects will help students apply AR knowledge to the real world and further develop collaboration skills, building a portfolio for potential careers in AR (or the so-called Metaverse), including gaming.

Connecting multiple courses such as computer graphics, computer vision, or user interface design, this course can greatly accompany them to also develop systems thinking skills. However, knowledge from these courses are not required.

Course Topics

• Introduction to augmented reality, Unity, and Vuforia
• Displays
• Image targets and transformations
• Tracking
• Sounds, animations
• Visual coherence: lighting, shadows, occlusions
• Situated visualization
• Interaction techniques
• Physics
• Plane finding
• Modeling

Learning Outcomes

At the end of the course, you will be able to:

• Define what is AR, compare different types of AR displays, and understand the capabilities and applications of AR
• Explain the graphics pipelines of AR systems, i.e., model, view, and perspective transformations
• Explain image target tracking computer vision algorithms and develop applications using the Vuforia engine framework, and understand nonvisual tracking methods using a variety of sensors
• Design, develop and deploy AR applications using the Unity framework, such as modeling, animating, and manipulating AR objects, simulating physics of real objects, adding sounds, ensuring visual coherence by adding lighting and occlusions, adding virtual buttons, annotating physical objects, and deploying to smartphones
• Develop situated visualizations, understand the pipeline, challenges, x-ray visualization to reveal hidden objects, and select AR visualization techniques to declutter a scene,
• Select input and output modalities for AR object interactions and user interfaces

Assessments

The assessments for this course include the following:

Three programming assignments	50%
Short homework exercises	15%
Technical topic investigation*	10%
Final project*	25%

*For graduate students the requirements are more challenging for the final project and the technical topic investigation. This is the main differentiating factor between the undergraduate and graduate versions of this class.

Final Grades

Final grades will be determined using a straight scale as shown below.

Symbol	Interval (%)
A	[93, 100]
A-	[90, 93)
B+	[87, 90)
B	[83, 87)
B-	[80, 83)
C+	[77, 80)
C	[73, 77)
C-	[70, 73)
D+	[67, 70)
D	[63, 67)
D-	[60, 63)
F	[0, 60)

Required Text

The required textbook is Augmented Reality: Principles and Practice, by Dieter Schmalstieg and Tobias Höllerer, Addison-Wesley, 2016. ISBN-10: 0321883578; ISBN-13: 9780321883575.

There is a companion website at www.augmentedrealitybook.org. The authors also have a curated playlist of recent AR development on YouTube.

Resources for locating the textbook and reference materials include the Mines Official Bookstore and Arthur Lakes Library.

Policies and Campus Resources

The COVID Smart Classroom

As new variants of COVID-19 continue to evolve, all campus community members are asked to make thoughtful choices about their health and be mindful that those choices will affect our whole community.

• Anyone experiencing COVID-19 symptoms should wear a mask, not report to work or attend classes, and get tested as soon as possible.
• Anyone experiencing respiratory symptoms – even after a negative COVID test – should remember there are many respiratory viruses circulating in our community. Please be considerate of others and wear a mask whenever you have any cold-like symptoms.
• Masking is one of the most effective ways to protect yourself and others, especially in indoor settings when community transmission levels are high. Please continue to respect an individual’s decision to wear a mask even if it is not required. Masks are still required in the Student Health Center.
• Wash your hands frequently using soap and water or hand sanitizer.

If you test positive for COVID-19 (rapid antigen or PCR test):

• Stay home for a minimum of five (5) days and isolate yourself from others at home, as directed by the CDC.
• Communicate with your professors via email or the excused absence form that you are ill or have tested positive for COVID-19. You should communicate this 5 day absenteeism to you faculty OR through the Student Life Excused Absence Process. Students can complete the Excused Absence here: mines.edu/student-life/forms/personal-excused-absence/.
• For additional support, please reach out to one of the many resources on campus:
• Academic support: Email CASA at casa@mines.edu.
• Student Life Office: Excused absence (only if you must be away from class for more than three days).
• Experiencing a mental/emotional challenge? Email Student Outreach & Support (SOS) at care@mines.edu or fill out a SOS Referral.

Regarding Absences

The Mines default position is to trust students unless they give us a reason not to. Please think carefully before attempting to abuse this trust. As long as your number of short-term illnesses is low, we will not require anything beyond the answers to the questions below to mark you as excused, and you can work through what you missed for your own understanding. However, if you end up requesting a large number of excused absences, suspicious absences, or opportunistic absences, then we may follow-up and require you to submit proof that you have worked through the missed material before excusing you (in part, to make sure you are not falling behind). If we determine that you are violating the intent of this policy, then we will treat it as Academic Misconduct or a Code of Conduct matter.

Information to provide in email to instructor:

1. Name and CWID
2. Course Name
3. Section and/or Time
4. Class day missed
5. Do you certify that you are indeed not feeling well and that after submitting this request will be limiting your activity to recover and/or lessen exposure to others as much as possible until you feel better?

Diversity and Inclusion

At Colorado School of Mines, we understand that a diverse and inclusive learning environment inspires creativity and innovation, which are essential to the engineering process. We also know that in order to address current and emerging national and global challenges, it is important to learn with and from people who have different backgrounds, thoughts, and experiences.

Our students represent every state in the nation and more than 90 countries around the world, and we continue to make progress in the areas of diversity and inclusion by providing Diversity and Inclusion programs and services to support these efforts.

Disability Support Services

The Colorado School of Mines is committed to ensuring the full participation of all students in its programs, including students with disabilities. If you anticipate or experience any barriers to learning in this course, please feel welcome to discuss your concerns with me. Students with disabilities may also wish to contact Disability Support Services (DSS) to discuss options to removing barriers in this course, including how to register and request official accommodations. Please visit their website at disabilities.mines.edu for contact and additional information. If you have already been approved for accommodations through DSS, please meet with me at your earliest convenience so we can discuss your needs in this course.

Accessibility within Canvas

Read the Accessibility Statement from Canvas to see how the learning management system at the Colorado School of Mines is committed to providing a system that is usable by everyone. The Canvas platform was built using the most modern HTML and CSS technologies, and is committed to W3C’s Web Accessibility Initiative and Section 508 guidelines.

Discrimination, Harassment, and Title IX

All learning opportunities at Mines, including this course, require an environment that allows each student to be able to learn without fear of discrimination or harassment based on any protected class. Mines’ core values of respect, diversity, compassion, and collaboration will be honored in this course, and the standards in this class are the same as those expected in any professional work environment. (More information can be found here.) Discrimination or harassment of any type, including sexual harassment, sexual assault, dating violence, domestic violence, and stalking, are prohibited under the Policy Prohibiting Unlawful Discrimination and the Policy Prohibiting Sexual Harassment, Sexual Assault, and Interpersonal Violence. As a participant in this course, we expect you to respect your instructor and your classmates. As your instructor, it is my responsibility to foster a learning environment that supports diversity of thoughts, perspectives and experiences, and honors your identities. To help accomplish this:

1. Course rosters are provided to the instructor with the student’s legal name. I will honor your request to address you by a preferred name and I will use your identified pronouns. Please advise me of this preference early in the semester so that I may make appropriate changes to my records.
2. If something is said or done in this course (by anyone, including myself) that made you or others feel uncomfortable, or if your performance in the course is being impacted by your experiences outside of the course, please report it to me (if you are comfortable doing so), to the mines.edu/institutional-equity-title-ix/reporting/, and to mines.edu/institutional-equity-title-ix/submit-report/ (an anonymous option).

In this course, we will cultivate a community that supports survivors, prevents interpersonal violence, and promotes a harassment free environment. Title IX and Colorado State law protects individuals from discrimination based on sex and gender in educational programs and activities. Mines takes this obligation seriously and is committed to providing a campus community free from gender and sex-based discrimination. Discrimination, including sexual harassment, sexual violence, dating violence, domestic violence, and stalking, is prohibited and will not be tolerated within the Mines campus community. If these issues have affected you or someone you know, you can access the appropriate resources here: mines.edu/title-ix/. You can also contact the Mines Title IX Coordinator, Carole Goddard, at 303.273.3260 or titleix@mines.edu for more information. It’s on us, all of the Mines community, to engineer a culture of respect.

Preferred First Name Project

In order to foster a more inclusive environment for students and faculty who use Canvas to teach, learn and collaborate, ITS is implementing the use of preferred first names throughout the entire Canvas learning management system at Mines. This change will transition Canvas to fully utilize preferred first names regardless of feature, function, or user. You can add a preferred first name by following the steps on the Preferred First Name website.

Why Are Preferred First Names important? Calling a person by their preferred name shows respect. Using preferred names contributes to the University’s goal of providing an empowering, safe and nondiscriminatory educational and work environment. Someone’s name is an extremely important part of a person’s identity. We encourage everyone to utilize a person’s preferred name and pronouns whenever addressing or referring to them.

CARE @ Mines

If you feel overwhelmed, anxious, depressed, distressed, mentally or physically unhealthy, or concerned about your wellbeing overall, there are resources both on- and off-campus available to you. If you need assistance, please ask for help form a trusted faculty or staff member, fellow student, Dean of Student Office (Raise Your Hand), or any of the resources below. As a community of care, we can help one another get through difficult times. If you need help, reach out. If you are concerned for another student, offer assistance and/or ask for help on their behalf. Students seeking resources for themselves or others should visit care.mines.edu.

Additional suggestions for referrals for support, depending on comfort level and needs include:

• CARE at Mines: care.mines.edu for various resources and options, or to submit an online “CARE report” about someone you’re concerned about, or email care@mines.edu
• CASA – mines.edu/casa/ for academic advising, tutoring, academic support, and academic workshops
• Counseling Center – mines.edu/counseling-center/ or students may call 303-273-3377 to make an appointment. There are also online resources for students on the website. Located in the Wellness Center 2^nd floor. Located at 1770 Elm St.
• Health Center – mines.edu/student-health/ or students may call 303-273-3381 for appointment. Located in Wellness Center 1^st floor.
• Colorado Crisis Services – For crisis support 24 hrs/7 days, either by phone, text, or in person, Colorado Crisis Services is a great confidential resource, available to anyone. coloradocrisisservices.org , 1-844-493-8255, or text “TALK” to 38255. Walk-in location addresses are posted on the website.
• Food and/or Housing – Any student who faces challenges securing their food or housing and believes this may affect their performance in the course is urged to contact the Dean of Students for support. Furthermore, please notify your professor if you are comfortable in doing so. This will enable your professor to provide resources that may be available.

All of these options are available for free for students. The Counseling Center, Health Center, and Colorado Crisis Services are confidential resources. The Counseling Center will also make referrals to off-campus counselors, if preferred.

In an emergency, you should call 911, and they will dispatch a Mines or Golden PD officer to assist.

Center for Academic Services and Advising (CASA)

CASA (mines.edu/casa) provides a variety of services to support students during their time at Mines.

• Academic Advising: All students are advised by Academic Advising Coordinators in CASA throughout their undergraduate studies at Mines.
• Academic Support Services: CASA offers a wide variety of support services designed to assist students throughout their undergraduate degree. Examples include pre-finals workshops, major exploration events, and the specific support services listed below.
  • Tutoring: Tutoring services are offered for all core curriculum courses and many major courses by peers. Tutoring is offered Sunday through Thursday in CASA (Aspen Hall), the Library, and via zoom.
  • Core Review Sessions: Core Review Sessions are group review sessions held by a peer facilitator before common core course exams. Peer facilitators also host regular office hours for more individualized assistance.
  • Academic Coaching: Students can work with CASA Advisors to develop the skills and technique of studying well in college, such as test-prep and cognitive learning development, in a one-on-one setting.
  • Faculty in CASA: Faculty from various departments host their regular office hours in CASA. Students are encouraged to utilize these professors for assistance with material and/or questions on course planning.

The Writing Center

The writing center is a free academic support service available to all members of the campus community including undergraduate and graduate students. We can assist you at any stage of the writing process, from brainstorming to final revisions. You do not need a complete draft to make an appointment. Our consultants are experts in a variety of composition and communication fields, providing support as you work on projects such as lab reports, essays, collaborative papers, scholarly publications, thesis chapters, and oral presentations. Whether you are focusing on organization or sentence structure, the Writing Center can evaluate your individual needs and tailor each appointment so that you become a more effective and efficient communicator. The Writing Center is open Sunday-Friday for in-person and online appointments. To learn more about our services and to make an appointment, please visit writing.mines.edu. For questions, please e-mail writing@mines.edu.

Policy on Academic Integrity/Misconduct

The Colorado School of Mines affirms the principle that all individuals associated with the Mines academic community have a responsibility for establishing, maintaining an fostering an understanding and appreciation for academic integrity. In broad terms, this implies protecting the environment of mutual trust within which scholarly exchange occurs, supporting the ability of the faculty to fairly and effectively evaluate every student’s academic achievements, and giving credence to the university’s educational mission, its scholarly objectives and the substance of the degrees it awards. The protection of academic integrity requires there to be clear and consistent standards, as well as confrontation and sanctions when individuals violate those standards. The Colorado School of Mines desires an environment free of any and all forms of academic misconduct and expects students to act with integrity at all times.

Academic misconduct is the intentional act of fraud, in which an individual seeks to claim credit for the work and efforts of another without authorization or uses unauthorized materials or fabricated information in any academic exercise. Student Academic Misconduct arises when a student violates the principle of academic integrity. Such behavior erodes mutual trust, distorts the fair evaluation of academic achievements, violates the ethical code of behavior upon which education and scholarship rest, and undermines the credibility of the university. Because of the serious institutional and individual ramifications, student misconduct arising from violations of academic integrity is not tolerated at Mines. If a student is found to have engaged in such misconduct sanctions such as change of a grade, loss of institutional privileges, or academic suspension or dismissal may be imposed.

The complete policy can be found in the Mines’ Policy Library.

Grading Policy

If you face uncontrollable, unusual, or extreme challenges and are able to provide appropriate documentation, I am happy to make reasonable accommodations for you to make up a missed quiz, lab, or homework assignment.

Extra credit may be offered for additional learning activities related to this class. Unless otherwise detailed in this syllabus, the awarding of extra credit is at the discretion of the instructor and is not guaranteed.

Coursework Return Policy

The course staff will try its best to grade and return assignments to you within 2 weeks of submission, along with suitable materials/feedback that enable students to understand how to improve their learning/performance.

Course Issues and Concerns

As part of good professional practice, students are encouraged to speak with the faculty directly to raise issues and concerns with regard to the course professionally in compliance with the student code of conduct. Students can also reach out to the course coordinator Dr. Tom Williams or the head of the Department of Computer Science Dr. Iris Bahar the course is being offered through. The department head can investigate and work with the faculty member to resolve course-related concerns. Students’ final point of contact is the Dean of Energy and Materials Programs, Dr. John Berger, who can make any final decisions.

Participating in this Course

Expectations for Participation

You are expected to engage in all course activities, tasks, and assignments as an emerging professional.

You are expected to spend between TBD-TBD hours on this course each week during out-of-class time, in addition to actively participating during the class-time each week.

Profile in Canvas

As part of the learning experience at the Colorado School of Mines, our class will be utilizing online learning resources and experiences through the Canvas learning management system. In order to help build community in this online learning environment, you are encouraged to upload your profile picture to Canvas. Photos should be similar to the photos taken for passports or state identification cards.

Expectations of online etiquette or netiquette

Here are few do’s and don’ts about communicating in your course through emails or in online discussion forums:

• Do…
  • Ask questions and engage in conversations as often as possible—feel free to contact the instructor via the discussion forum for questions or via email or other communication.
  • Be patient and respectful of others and their ideas and opinions they post online.
  • Remember to be thoughtful and use professional language. Keep in mind that things often come across differently in written text, so review your writing before posting.
  • Be prepared for some delays in response time, as “virtual” communication tends to be slower than “face-to-face” communication.
  • Contact the instructor if you feel that inappropriate content or behavior has occurred as part of the course.

• Do NOT…
  • Use inappropriate language—this includes, but is not limited to, the use of curse words, swearing, or language that is derogatory.
  • Post inappropriate materials—for example, accidentally posting/showing a picture that is not appropriate for the course content.
  • Post in ALL CAPS, as this is perceived as shouting and avoid abbreviations and informal language (“I’ll C U L8R”).
  • Send heated messages even if you are provoked. Likewise, if you should happen to receive a heated message, do not respond to it.
  • Send an email or post to the entire class, unless you feel that everyone must read it.

Course Schedule

Note that we use Canvas for

• course content (Modules),
• submissions (Assignments),
• grading (Grades),
• and communication (Discussions).

---

Week	Date	Topics	Readings	Handouts	Due
1	8/23 (T)	Course Intro Intro to AR	Chapter 1 (free sample chapter)	HW1: Administrative stuff (Due 8/25) HW2: AR killer apps (Due 8/25) Dr. Zhao Han’s Research on AR
1	8/25 (Th)	Unity: Intro		HW3: Roll-a-Ball (Due 8/30)	HW1, HW2
2	8/30 (T)	AR Displays Transformations	Chapter 2	HW4: “Use cases” for displays (Due 9/1)	HW3
2 (Sep)	9/1 (Th)	Vuforia: Intro		USB webcam (return 12/1) Printed image target HW5: Clock animation (Due 9/6) A1: Augmenting Real Objects (Due 9/20)	HW4 (peer review due next day)
3	9/6 (T)	Image Targets Vuforia: Image Targets	Chapter 3 (3.1, 3.2, 3.6) Chapter 4 (4.1, 4.3, skim algorithms) Chapter 5	HW6: Vuforia image target (Due 9/8)	HW5
3	9/8 (Th)	Tracking Vuforia: Tracking Unity: Sounds Unity: Models Unity: Recorder	Chapter 4 (4.4-4.6, skim algorithms)		HW6
4	9/13 (T)	Non-visual Tracking and Sensor Fusion Unity: Animation	Chapter 3 (3.3-3.5, 3.7)	HW7: Animation (Due 9/15)
4	9/15 (Th)	Visual Coherence Occlusions Lighting Shadows Other Effects Unity: Occlusions	Chapter 6		HW7
5	9/20 (T)	Unity: Lighting, Shadows Assigning Magic Leap 1 devices		Magic Leap 1 devices HW8: Commercial app visual coherence (Due 9/22) A2: Visual Coherence (Due 9/29)	A1
5	9/22 (Th)	Situated Visualization Challenges Visualization registration Annotations and labeling X-ray visualization Spatial manipulation Information filtering	Chapter 7	HW9: Hyper-Reality (Due 9/27) Technical Topic Proposal (Due 10/6)	HW8
6	9/27 (T)	Magic Leap 1 Deployment Android and iOS Deployment			HW9
6	9/29 (Th)	Interaction 1 Input Modalities Output Modalities	Chapter 8 (8.1-8.2)	HW10: Deployment Final Project Proposal (Due 10/13)	A2
7 (Oct)	10/4 (T)	Midcourse Feedback and Reflection Unity: Selection Unity: Manipulation Interaction 2	Chapter 8 (8.3-8.9)	HW11: Select an object (Due 10/11)	HW10
7	10/6 (Th)	Unity: Physics Vuforia: Physics		Technical Topic Presentation (Due 10/27-11/10)	Technical Topic Proposal
8	10/11 (T)	Vuforia: Virtual Buttons Evaluation		A3: Interaction (Due 10/25) HW12: App brainstorm (Due 10/20)	HW11
8	10/13 (Th)	Discuss app ideas from brainstorm session			Short App Brainstorm Talk (HW12) Final Project Proposal
9	10/18 (T)	No Class (Fall Break)
9	10/20 (Th)	Vuforia: Plane Modeling and Annotation Plane Fitting	Chapter 9	Schedule of Presentations
10	10/25 (T)	Unity: AR Foundation Unity: Location Services		Team Policies Agreement (Due 10/27)	A3
10	10/27 (Th)	Technical Topic Presentation			Team Policies Agreement
11 (Nov)	11/1 (T)	Technical Topic Presentation		Final Project Checkpoint 1
11	11/3 (Th)	Technical Topic Presentation		HW13: Guest Speaker 1 (Due 11/10)
12	11/8 (T)	Technical Topic Presentation		HW14: Guest Speaker 2 (Due 11/15)	Final Project Checkpoint 1
12	11/10 (Th)	Guest Speaker 1: Dr. Tom Williams on AR in Human-Robot Interaction		Final Project Teaser (Due 11/22)	Technical Topic Presentation Q&A HW13
13	11/15 (T)	Guest Speaker 2: Dylan Fox on XR Accessibility		Final Project Checkpoint 2	HW14
13	11/17 (Th)	No lecture today, TA in-class help with projects
14	11/22 (T)	Final Project Teaser Presentation		Final Project Demo (Due 11/29 & 12/1) and Reviews (Due 12/1) Final Project Report (Due 12/6)	Final Project Teaser (in class) Final Project Checkpoint 2
14	11/24 (Th)	No Class (Thanksgiving Break)
15	11/29 (T)	Final Project Demo		Final Project Demo Schedule (Posted on Monday)	Webcam/Magic Leap 1 devices return
15 (Dec)	12/1 (Th)	Final Project Demo			Final Project Reviews Devices return before leaving
	12/6 (T)	No lecture today, work on final project report			Final Project Report

Academic Calendar 2022-2023

Acknowledgement

The course material was updated and revised based on the textbook website and from Emeritus Associate Professor William Hoff.

---

This course was approved on April 20, 2022.