During my Bachelor's studies, when I was checking out various IT industries, I was most interested in computer vision. Last year, I did an internship working in this area at a start-up company where I monitored a warehouse, and when I was selecting the topic for my Bachelor's diploma thesis, I thought I could use the knowledge I gained there. Moreover, it seemed it could have real-world applications. So I contacted Professor Drahanský, who suggested tracking the movement of visitors in museum exhibitions as my thesis topic. That made me even more excited and I knew I wanted to tackle this issue. Martin Drahanský became my thesis supervisor and together with Tomáš Dyke they helped me a great deal. It was them who suggested I entered the Excel@FIT competition.

The principle is similar to, for example, the monitoring of website traffic: there you also deal with the placement and appearance of different elements to keep people's attention for as long as possible. In the same way, museums and galleries need to think about the placement of their exhibits. That's why they keep track of where people go, how much time they spend there and which exhibits attract the most attention. My supervisor provided me with literature from which I learned that the most common current tracking technique still relies on pencil and paper: researchers in the study area manually record visitor movements. Another method of tracking uses radio technology. The visitor carries a locator that indicates his position. This can be, for example, an RFID locator or a smartphone with a special app. Access points are placed in the monitored area and the location of the access point is calculated based on the received WiFi signal strength from each point. This solution, however, requires complex technology and the deployment of special beacons to receive these signals, so it adds up to quite a demanding investment in infrastructure.

Our system works with camera technology: using neural networks, we find out where the visitor is and, by configuring the camera, we can pinpoint their location within the monitored exhibit. Based on the position and image, we can identify the visitor and track him or her by counting the person's trajectories. I record the data in a histogram, which shows the distribution of clothing colour and location of a particular person. Thanks to these two parameters, visitors can be tracked quite accurately throughout a building. Then I calculate statistics for individual exhibits. The museum management identifies the exhibits they are interested in, and I draw up statistics showing how many people stopped by and how long they spent there. In this way, we actually evaluate the attractiveness of the exhibits and provide valuable feedback on their optimal placement and exhibition composition.

Have you tested the system in live operation?

So far, I've been doing testing on my own dataset showcasing the output itself, which tracks the trajectory of people and produces a heat map. Further testing was conducted on the wildtrack dataset, which was developed at a Swiss university and provides synchronised footage from up to seven cameras scanning the square in front of the university. A single recording has about twenty people on it and provides 400 images, which makes for a total of about 56,000 detections. I also performed comparisons with other methods on this dataset. As far as detection is concerned, my algorithm did very well. Thre was some room for improvement in tracking, but compared to existing methods, my algorithm tracked the persons for longer.

First of all, I designed it to be modular. It is composed of several parts that are easily interchangeable. Although I tested and compared the performance of the systems that processed my own dataset and the wildtrack dataset, only one module needed to be replaced. This makes changing the neural networks for detecting people or the algorithm for tracking people really flexible and simple, while the rest of the system still works the same. Changes to the algorithm will be easy to implement, and this is where the high future potential of my system lies.

Yes, due to its modularity, the system is able to accommodate certain changes that each particular site will require. The first application site is the National Museum in Prague, which independently identified the need to track the visitors. My Bachelor's thesis was based on that - it was clear that they wanted to use the system and that I was programming something that had to work in the real world. Now I have a final exam to pass at the end of the 3rd year, but I've made arrangements that I will do some more testing at the Faculty right afterwards and then proceed to the installation.

I would also like to continue working at the startup company I trained at last summer to further improve my computer vision expertise. I'm already enrolled in a Master's program in machine learning. Although it is very closely related, I am still considering switching my major to computer vision. Anyway, I'll be spending a lot of time in Prague this summer getting my system up and running.