I created a very simple mobile phone app in Flutter. It would allow you to take a picture, which it sent off to a Docker stack with a Python Image Segmentation script. The script detected windows in the photo and returned it with the windows highlighted.

I created the model for detection through Tensorflow 2.0. The model was trained on a set of 1000 images which had their windows highlighted. I created the tagging tool for this and we employed local students to do the tagging for us. We later upgraded the model to detect doors and roofs.

The model training process is fairly complex, but I was able to train the system correctly. With enough training images, the tool can be taught to detect any kind of object within an image (faces, logos etc). The images that we asked the students to tag came from a Google Streetview scrape tool I created.