Article | March 15, 2021
A few weeks ago, I created a Tensorflow model that would take an image of a street, break it down into regions, and - using a convolutional neural network - highlight areas of the image that contained vehicles, like cars and trucks.
I called it an image classification AI, but what I had really created was an object detection and location program, following a typical convolutional method that has been used for decades - as far back as the seventies.
By cutting up my input image into regions and passing each one into the network to get class predictions, I had created an algorithm that roughly locates classified objects. Other people have created programs that perform this operation frame by frame on real time video, allowing computer programs to draw boxes around recognised objects, understand what they are and track their motion through space.
In this article, I’ll give an interesting introduction to object detection in real-time video. I’ll explain why this kind of artificial intelligence is important, how it works, and how you can implement your own system with Yolo V3. From there, you can build a huge variety of real time object tracking programs, and I’ve included links to further material too.
Importance of real-time object detection
Object detection, location and classification has really become a massive field in machine learning in the past decade, with GPUs and faster computers appearing on the market, which has allowed more computationally expensive deep learning nets to run heavy operations. Real time object detection in video is one such AI, and it has been used for a wide variety of purposes over the past few years.
In surveillance, convolutional models have been trained on human facial data to recognise and identify faces. An AI can then analyse each frame of a video and locate recognised faces, classifying them with remarkable precision.
Real-time object detection has also been used to measure traffic levels on heavily frequented streets. AIs can identify cars and count the number of vehicles in a scene, and then track that number over time, providing crucial information about congested roads.
In wildlife, with enough training data a model can learn to spot and classify types of animals. For example, a great example was done with tracking racoons through a webcam, here. All you need is enough training images to build your own custom model, and such artificial intelligence programs are actively being used all around the world.
Background to Yolo V3
Until about ten years ago, the technology required to perform real-time object tracking was not available to the general public. Fortunately for us, in 2021 there are many machine learning libraries available and practically anyone can get started with these amazing programs.
Arguably the best object detection algorithms for amateurs - and often even professionals - is You Only Look Once, or YOLO. This collection of algorithms and datasets was created in the 2000s and became incredibly popular thanks to its impressive accuracy and speed, which lends it easily to live computer vision.
My method for object detection and recognition I mentioned at the start of this article happens to be a fairly established technique. Traditional object recognition would split up each frame of a video into “regions”, flatten them into strings of pixel values, and pass them through a deep learning neural network one by one. The algorithm would then output a 0 to 1 value indicating the chance that the specific region has a recognized object - or rather, a part of a recognized object - within its bounds.
Finally, the algorithm would output all the regions that were above a particular “certainty” threshold, and then it would compile adjacent regions into bounding boxes around recognized objects. Fairly straightforward, but when it comes down to the details, this algorithm isn’t exactly the best.
Yolo V3 uses a different method to identify objects in real time video, and it’s this algorithm that gives it its desirable balance between speed and accuracy - allowing it to fairly accurately detect objects and draw bounding boxes around them at about thirty frames per second.
Darknet-53 is Yolo’s latest Fully Convolutional Network, or FCN, and is packed with over a hundred convolutional layers. While traditional methods pass one region at a time through the algorithm, Darknet-53 takes the entire frame, flattening it before running the pixel values through 106 layers. They systematically split the image down into separate regions, predicting probability values for each one, before assembling connected regions to create “bounding boxes” around recognized objects.
Luckily for us there’s a really easy way we can implement YoloV3 in real time video simply with our webcams; effectively this program can be run on pretty much any computer with a webcam. You should note however that the library does prefer a fast computer to run at a good framerate. If you have a GPU it’s definitely worth using it!
The way we’ll use YoloV3 is through a library called ImageAI. This library provides a ton of machine learning resources for image and video recognition, including YoloV3, meaning all we have to do is download the pre-trained weights for the standard YoloV3 model and set it to work with ImageAI. You can download the YoloV3 model here. Place this in your working directory.
We’ll start with our imports as follows:
import numpy as np
from imageai import Detection
Of course, if you don’t have ImageAI, you can get it using “pip install imageai” on your command line or Python console, like normal. CV2 will be used to access your webcam and grab frames from it, so make sure any webcam settings on your device are set to default so access is allowed.
Next, we need to load the deep learning model. This is a pre-trained, pre-weighted Keras model that can classify objects into about a hundred different categories and draw accurate bounding boxes around them. As mentioned before, it uses the Darknet model. Let’s load it in:
modelpath = "path/yolo.h5"
yolo = Detection.ObjectDetection()
All we’re doing here is creating a model and loading in the Keras h5 file to get it started with the pre-built network - fairly self-explanatory.
Then, we’ll use CV2 to access the webcam as a camera object and define its parameters so we can get those frames that are needed for object detection:
cam = cv2.VideoCapture(0)
You’ll need to set the 0 in cv2.VideCapture(0) to 1 if you’re using a front webcam, or if your webcam isn’t showing up with 0 as the setting. Great, so we have imported everything, loaded in our model and set up a camera object with CV2. We now need to create a run loop:
ret, img = cam.read()
This will allow us to get the next immediate frame from the webcam as an image. Our program doesn’t run at a set framerate; it’ll go as fast as your processor/camera will allow.
Next, we need to get an output image with bounding boxes drawn around the detected and classified objects, and it’ll also be handy to get some print-out lines of what the model is seeing:
img, preds = yolo.detectCustomObjectsFromImage(input_image=img,
As you can see, we’re just using the model to predict the objects and output an annotated image. You can play around with the minimum_percentage_probability to see what margin of confidence you want the model to classify objects with, and if you want to see the confidence percentages on the screen, set display_percentage_probability to True.
To wrap the loop up, we’ll just show the annotated images, and close the program if the user wants to exit:
if (cv2.waitKey(1) & 0xFF == ord("q")) or (cv2.waitKey(1)==27):
Last thing we need to do outside the loop is to shut the camera object;
And that’s it! It’s really that simple to use real time object detection in video. If you run the program, you’ll see a window open that displays annotated frames from your webcam, with bounding boxes displayed around classified objects.
Obviously we’re using a pre-built model, but many applications make use of YoloV3’s standard classification network, and there are plenty of options with ImageAI to train the model on custom datasets so it can recognize objects outside of the standard categories. Thus, you’re not sacrificing much by using ImageAI.
Good luck with your projects if you choose to use this code!
Yolo V3 is a great algorithm for object detection that can detect a multitude of objects with impressive speed and accuracy, making it ideal for video feeds as we showed on the examples aboves.
Yolo v3 is important but it’s true power comes when combined with other algorithms that can help it process information faster, or even increasing the number of detected objects. Similar algorithms to these are used today in the industry and have been perfected over the years.
Today self-driving cars for example will use techniques similar to those described in this article, together with lane recognition algorithms and bird view to map the surroundings of a car and pass that information to the pilot system, which then will decide the best course of action.
Article | June 18, 2020
When you first got your business off the ground, you may or may not have paid much attention to the technologies that would be available to you in the years to come—like machine learning. Machine learning was the stuff of science fiction just decades ago; now it’s practically everywhere. So, what is machine learning? Simply put, machine learning is a subset of artificial intelligence in which computer algorithms learn from large datasets in order to make more accurate predictions over time. Obviously, it’s a lot more complicated than that, but it poses numerous benefits to business owners—assuming it’s used the right way. Here are five tips for successfully adopting machine learning technologies in your day-to-day operations.
Article | April 12, 2021
There is nothing new about fake news. It has been in existence for centuries, albeit without the scaffolding of support from social media. From housewives’ tales to gossip magazines, the Trojan horse to the misinformation around the D-Day landing site, fake news has been a rite of passage.
The Russian military made this into a fine art with “maskirovka,” the doctrine gaining superiority through deception, denial and disinformation. However, it was the 2016 U.S. presidential election that branded it with a legit identity and with such alacrity that today, I find myself questioning everything I read or hear about, no matter the veracity of the source. Fake news is a contagion that has the potency to be as disruptive as the coronavirus and must be fought with equal urgency.
If you cannot solve the problem, manage it.
The power behind fake news is big data — the quantum of data generated and its velocity of distribution. Big data feeds companies with interesting consumer insights on evolving trends and behaviors, which are then beautifully packaged into text, video or audio content by harnessing machine learning and deep learning algorithms.
The slips happen here. If I were to personify fake news, Cersei Lannister, the manipulative, power-hungry queen in Game of Thrones, would be the perfect candidate. Cersei embellishes the truth with dramatic twists and turns to create compelling lies. We experienced a similar situation when news broke that President Trump’s grandfather owned the Arctic Restaurant and Hotel in Bennett, British Columbia, during the 1890s and 1900s, which fueled an interesting twist on the source of the family's wealth.
While AI will help us identify fake news, we need a preventive measure that nips it in the bud
It is almost difficult to differentiate fake news from real news. While AI will help us identify fake news, we need a preventive measure that nips it in the bud — a vaccination rather than medication.
If tech helps in creating an issue, should tech help solve it too?
Based on my years of experience in implementing these solutions for large enterprises and developing next-gen blockchain offerings with startups, I believe blockchain may just be the remedy we are looking for. Most technologists, however, do not consider blockchain to be a relevant or credible technology, with the primary criticisms being its lack of widespread adoption and its esotericism. But I believe the contrary. The vision of grandma-proof blockchain is becoming real — to create an inclusive global, scalable blockchain solution that can cater to every human need.
Blockchain should be our weapon to effectively reduce and ultimately eradicate fake news.
In blockchain, no single individual or group holds the authority, but everyone needs to approve; therefore, it enables the highest degree of integrity, privacy and security
Blockchain is nothing but a distributed ledger that helps build trust in decentralized networks and that runs on the computing power of its participants. No single individual or group holds the authority, but everyone needs to approve; therefore, it enables the highest degree of integrity, privacy and security. This is accomplished by consensus algorithms. Each blockchain has adopted some form of it, and some even claim to have consensus that can prevent obfuscation of the truth even when faced with over 90% malicious intent.
Blockchain technology enables a "shared single version of truth" across multiple entities based on two fundamental characteristics: immutability and traceability.
Immutability is when a blockchain ledger has the capability to remain unaltered, effectively ensuring that any data on the blockchain cannot be altered — only built upon.
Each block created has a unique identity and timestamp attached to it that builds a fortress around the data. Innovative upcoming blockchains use crypto-biometric identity to further buttress the fort. For example, Mediachain, a decentralized independent music library, uses blockchain to protect the originator’s authenticity by providing information about the creator, producer and lyrics to listeners. Steemit is a decentralized social media site that rewards content creators who also interact with other users. Each content piece or interaction is recorded on the immutable record by blockchain.
And if news companies were to adopt blockchain — and organizations like the New York Times are already working on this — this is what we might expect:
Journalists could create a block (an entry in a distributed ledger) and upload news via text, image or video.
Editors would then create another new block with an edited version of the news, leaving the original block unchanged.
Publishers (news agencies) would then publish the news based on their block and any changes that they might make.
Each one of the participants is authenticated on the blockchain with a simple touch of their finger while protecting the fidelity of the news.
Remember, entries cannot be changed, only built upon, and therefore, each change is recorded and allocated to a specific entity. For someone to “fake” the news, they would have to alter the data at each level. Infiltrating the high-security protocols would require considerable time and resource allocations.
As mentioned, each block that is created has a distinct identity attached, preferably a crypto-biometric for added security and individual control. So, if fake news is generated and circulated through social media using blockchain as the base, it becomes easier to pinpoint the culprit while establishing the real source of the news. This would ascribe true content ownership to credible creators.
Fake news creators are using advanced tech stacks to create deepfakes for digital deception. Generative adversarial networks (GANs) can help them to create deepfakes of images and videos that can even counteract or deceive advanced AI/ML algorithms. Of course, GANs are also being used to detect fake news now. If technology has helped fake news become compelling and believable, let’s use intelligent and available technology like blockchain to at least control it, if not eradicate it.
Then again, if blockchain had existed in the medieval ages, we would have been denied the entertaining antics of Cersei Lannister and the wonderful blockbuster series that kept most of us enthralled!
Article | December 20, 2020
Staying relevant and cutting edge in the business world is a struggle for businesses in any industry. Technology, including intelligent automation, is continually evolving. Businesses must change with it in order to be competitive and successful in our current macroeconomic world. The use of intelligent automation tools can help grow your business and improve how your business operates, reducing your operating costs while improving your production time.
Reducing Human Error
One of the most important benefits that intelligent automation brings to any business is the reduction of human error in the work place. People are naturally affected by their daily lives and outside influences. If a worker, for example, came into work tired or unwell, his or her job performance will likely suffer, the risk of human error becoming greater. Automation software cannot be affected by time of day, mood, lack of sleep, etc., allowing it to be completely consistent in performing the task it was programmed to do.
Additionally, humans need to be taught new tasks and require practice in order to master them, robotic process automation can be updated and perform the tasks instantly.
Max Yankelevich, founder & CEO of WorkFusion, says it best “Robots need only eight to 12 weeks to take over a back office function that humans take years to learn.”
In terms of business benefits, utilizing intelligent automation tools ensures performance consistency that will ultimately improve the overall quality of work, also allowing human workers to focus on higher priority and more important issues that require critical thinking.
Keeping Jobs Local
Employers have often ventured overseas to hire workers in other countries who can then perform basic tasks at a reduced wage, when compared to local employees. The bottom line can be better for these employers in the short-term, though working with outsourced employees means sending money overseas and trying to manage workers on another continent. Typically, over the long-term businesses that outsource overseas can experience unforeseen issues and costs due to the complications with depending on a foreign workforce.
With outsourced jobs being performed by intelligent automation tools businesses can focus on hiring skilled workers from the local market for the upper levels of the workforce.
Return on Investment
Perhaps the most intimidating factor in implementing intelligent automation within your business is the upfront cost. Putting money into something new is not a leap everyone wants to make. Intelligent automation, however, is not a gamble. Research shows that companies who use are able to automate around half of their tasks, increasing process time by fifty percent. Completing tasks more quickly means companies can take on more tasks without spending additional time on them. Depending on the industry, having jobs done quickly can mean increased revenue.
If performing redundant tasks quickly and accurately will not improve your company’s revenue, just simply utilizing intelligent automation tools certainly will. Such tools do not need pay, employee benefits, and can work overtime, the return of investment becomes apparent when considering all the expenses intelligent automation does not require.
Intelligent automation tools offer businesses unparalleled levels of productivity, efficiency, and value. Companies will want to avoid the risk of falling behind by adapting with the modern technology, the advantages of utilizing intelligent automation tools can lead companies to developing new business strategies they could have never even possibly conceived of previously.