Data and artificial intelligence – how they are leveraged to generate value today
01 | 2023 Ken Ikeda, Hands-on Azure / Solution Architect & Senior Consultant
Understanding the needs of the customer and end users is paramount. Do not embark on large-scale implementation projects or IT solutions until the benefits for the users are clearly defined. Keep in mind that achieving genuine business value may require streamlining your internal processes in addition to customizing systems. Only then can you begin building value-generating projects together.
Just 5 years ago, companies were dreaming about leveraging artificial intelligence and real-time data. Terms like deep learning, machine learning, and IoT platform were at the peak of the 2017 Gartner Hype Cycle. Companies were conducting various studies and experiments related to these topics.
While many succeeded in developing solutions that eased users' daily lives, for others, artificial intelligence and IoT remained mere experiments.
Today, the world shows that artificial intelligence and real-time data are widely utilized and integrated into our daily routines at work and at home. In many homes, a robot vacuum operates autonomously using real-time data. Many optimize their electricity consumption by tracking price fluctuations through Fingrid's mobile application.
Ken is Kipinä's seasoned data expert and hands-on Azure/solution architect with over 20 years of experience.
When talking about artificial intelligence, it is impossible not to praise ChatGPT. It is clear that AI-powered chatbots like ChatGPT will revolutionize the daily work of a software developer like myself. They already assist us in design, coding, and technical documentation. ChatGPT has only recently been released, so I can only imagine what it will be capable of in a few years.
When data guides users in the right direction, you are at the core of value creation. Examples of directions an organization might want to steer its end users toward include:
cost reduction
consolidating purchases into a single platform
recommendations to select a specific product
predicting when a product requires maintenance.
Data can also be used to steer organizational operations and streamline operating models by providing members with a real-time situational overview of the business and operational environment.
Such requirements place heavy demands on the technology platform. It must be capable of ingesting large volumes of real-time data and combining it with static data. Additionally, the platform must be able to analyze data and even leverage artificial intelligence, ultimately delivering data to its users almost in real time.
Traditionally, building, managing, and maintaining Big Data architectures has been challenging. Modern data architectures can be implemented in at least three different ways:
streaming systems
data lake
data warehouse.
Each approach has its own strengths, but implementing a unified data flow has always been challenging and complex. For example, combining real-time and batch data using a Lambda architecture practically means maintaining two separate systems.
Today, there are tools available for this purpose, and they can be applied using various architectural models. Tremendous development has taken place in this area as well, and modern architectures look quite different compared to 5–10 years ago.
However, good tools or architectures do not automatically guarantee that a sophisticated technical solution will generate value for the end user. Understanding user needs is paramount. Do not initiate large implementation projects before it is clear what the benefits to the users will be.
Ensure that every developer understands the business requirements and the true benefits to the end users. Prioritize information security and data protection so that users can utilize the service with peace of mind.
Keep in mind that achieving genuine business value may require streamlining your own processes in addition to customizing systems.
Please get in touch if you need sparring support or developers for demanding projects where data serving is at the heart of the user experience. At Kipinä, we specialise in demanding software development projects, leveraging data and AI, and understanding and helping customers. And finally, a little more about what the author likes to talk about and spar about:
One popular tool in the analytics world is Delta lake on top of Databricks. Databricks is an analytics platform based on the open source Spark. It is available on all major public cloud platforms, Azure, AWS and GCP. Delta Lake is an open source data warehousing solution that runs on top of Spark and provides features such as data integrity checks and data history management. Delta Lake on top of Databricks provides the ability to combine batch data with real-time stream data on a single platform, as well as AI and machine learning tools.