Meet Jakub Porzycki (Machine Learning Team Leader), a passionate and experienced AI and Machine Learning practitioner. Kuba uses his knowledge in business but also frequently and willingly shares it during educational events. See for yourself how much passion and commitment he has. 

1 . Kuba, you are our strong link in the context of AI and ML on many levels, as you are also eagerly involved in various development and recruitment initiatives. We’ll talk “technically” in a moment, but first, a big thank you to you for taking the time for this interview.  

Thank you for your kind words; the pleasure is mine. 🙂

2. So let’s get started. 😊 You’ve been active in the Machine Learning (ML) field for many years. When did you get the idea for such a career path? 

I spent several years working at AGH, where I was involved in research on computer simulations. This is also a kind of artificial intelligence, but the so-called symbolic one, where models are created differently than with machine learning. Therefore, when I decided to move to business, looking for a job in the AI area seemed natural. 

Looking from a broader perspective, I guess I’ve always wanted to do something with a “wow effect”. Something that is a bit like “magic”. Hence, I was interested first in computer science and programming, then scientific work, and finally artificial intelligence. Despite several years in the industry, it’s still fascinating when a model starts working and “just like that” gives the correct answers. 

3. What is ML (machine learning) to you? How do you understand this concept?  

When you think about it carefully, it’s a pretty tough question. ML is talked about in many ways, such as that it is a way of automating tasks, an extension of data analysis, a set of algorithms for “training” computers, or a method of acquiring business knowledge.  

In my opinion, it can be explained most simply by saying that Machine Learning is the automatic creation of models of the natural world from data. The model will be some mathematical description of the reality we create using ML algorithms and data. Of course, the model is always a simplification of reality.  

For example, by analyzing purchase history, we can build a model (e.g., a neural network). Based on the data, the neural network learns customers’ shopping preferences (i.e., the relationship between customer and product parameters) and encodes this information in the form of weights – connections between neurons in the network. With such a model, we can calculate what products a particular user will buy. 

4. How is ML used at Ailleron?

Ailleron creates ML models for banks, fintechs and leasing companies. We often develop models that operate on transactional data, such as transaction type classifiers, subscription classifiers, merchant classifiers, and recurring transaction detection. In addition to direct business value, such models also bring additional customer information that can be used in other models.   

In recent year (2023), we developed a number of solutions based on Generative AI and LLMs (Large Language Models). A good example is AI Prompter – a tool that helps customer center workers find relevant answers to customer’s questons. I know, it might seems like just another poor clone of ChatGPT, but I’m quite proud of what my team achieved in terms of processing complicated documents and preventing LLM’s halucinations.

We also offer solutions such as product recommendations, customer behavior prediction, anomaly detection, customer segmentation, or enhancements to chatbots. Of course, we are not limited to this list. Our company provides financial technology services, utilizing machine learning to solve the problems our clients and partners pose.  

5. Tell us what it looks like in practice. How to convince the client to implement the ML solution and show that it is proper and safe. 

When talking to clients, a lot depends on their awareness of Machine Learning. Until recently, many clients were quite distrustful of ML solutions. In conversations with them, it was crucial to show that machine learning allows us to solve some problems much cheaper than classical programming while having the ability to prove the high quality and reliability of models. Among other things, this is why we began to work intensively on model testing. In addition to the standard validation of models during training and tuning of hyperparameters, we introduced so-called “post-training testing,” a series of tests that verify the quality of the model from the business side. We have also established cooperation with NannyML, a company specializing in monitoring models after they have been put into production.  

Conversations are different when the client is fascinated by machine learning. Usually, in that situation, our client or partner already has an idea for the desired system. Our job then is to make sure the customer’s vision is feasible, possibly suggesting some changes in the approach. Unfortunately, the current hype for artificial intelligence, further strengthened after the release of ChatGPT, causes many people to approach ML as a “semi-magical” remedy for everything. 

And finally, there are those clients and partners who are very knowledgeable about Machine Learning. They usually already have their own ML teams and seek support on more challenging topics or new solutions to boost their results. We can gain the interest of such customers by showing them advanced solutions, e.g. in the area of: Generative AI and large language models, and the second is graph data representation and graph neural networks.

6. Algorithm selection, training, validation, testing, and deployment are all steps in the ML model. Can you briefly describe for us what these steps consist of? 

Algorithm selection is a decision made by us, the Machine Learning Engineers. We then choose the algorithm that will be used to train the ML model. This is important because we then make certain assumptions about the nature of the phenomenon we are modeling. For example, suppose I decide to use linear regression. In that case, the issue/ phenomenon can be modeled as a linear combination of inputs. Suppose I choose to use a recursive network. In that case, I expect that an important feature of the issue/ phenomenon is sequentiality, that is, that previous values affect subsequent values. Of course, many different ML algorithms can be tested during the work, but choosing the right ones shortens the work significantly.

On the other hand, model training is a process that happens automatically; ML algorithms train specific models based on the data provided to minimize the value of the loss function. Our task is to transform the data in advance into a form that will be easiest for the model to learn. Then, we choose the ML algorithm and the hyperparameters of a particular model we want to test. A hyperparameter could be, for example, the number of layers in a neural network, the number/type of neurons in a given layer, or the maximum depth of a decision tree. Then comes the time when we turn on the model training, the learning algorithm runs, and we can quietly go for coffee. Usually, we train multiple models with a different set of hyperparameters and choose the one with the lowest error function value.  

During validation, we check the quality of the model – usually only the best ones. Here, it is necessary to choose an appropriate measure of quality. Again, let me use an example: suppose we need to build a binary model that detects infrequent events, such as fraud occurring once every 1000 transactions. For each transaction, the model is supposed to return true if it thinks the transaction is fraud and false otherwise. If we take accuracy – the percentage of correct indications – as a measure of quality during validation, then a hopeless model that always returns false, meaning it detects no fraud, will have a staggering 99.9% efficiency.  It is worth noting that it is almost never possible to train a model that passes validation the first time, so we often return to the model training stage after validation. 

Once we have the model trained, we move on to detailed testing. We then analyze wrong answers to see if there are any systematic errors in the model’s performance. At Ailleron, we also test the model against business requirements (known as the post-training test). We verify such things as the correctness of the model on all base cases, like the lack of influence of business insignificant features on the model’s response (e.g., a person’s name does not influence the amount of credit offered) or the fulfillment of basic assumptions (e.g., the price of an apartment increases as the square footage increases). At this stage, the model is also checked for possible biases regarding social characteristics, known as social bias. 

Once the model passes all the tests, it can be prepared for deployment. This is, in practice, the packaging of the model with additional elements that allow communication with it. The model itself is simply saved as a file (usually containing a description of the structure and weights of the neural network). The model is wrapped in a microservice, which communicates via some interface (such as REST or Kafka), to work production-wise. To such a microservice, we can then send data, which must be in the same form we used to train the model. And receive a response. 

Came out a long answer. 🙂 But I think that’s it.  

7. What you are talking about is pretty complicated, but also fascinating. Tell us more about defining tasks effectively and understandably when creating a model so that communication with both the “technical” and “non-technical” person goes smoothly and efficiently.

Again, we return to a key aspect of our work, i.e., client interviews and workshops. From my experience, the initial focus is to talk with the “business” and answer some critical questions. First, you need to find out the business need the client wants to solve. Please note that this is about the customer’s actual problem rather than his idea of the ML. Perhaps a different approach than the one the customer has come up with will be better. It’s worth knowing his expectations and making sure they are realistic. In particular, whether the client realizes that no ML model will have 100% accuracy. It’s also important to know whether the client has data on which to teach the model and whether he can share it with us. Finally, probably most importantly, whether making an ML model solving such a problem is even possible.

Once you have positive answers to all of the above questions, you can move on to talking to technical people. Usually, these are the people responsible for providing the data. We can speak with them about the system from which the data will be provided (it must be the same system that will use the model later). It always takes much time to analyze the data schema and understand the different columns. It’s also worth making sure the data is current. Teaching the model on market data several years ago misses the point. 

8. Data is a crucial element in ML operations and development. How to acquire the most valuable ones?

Definitely, data is the most important thing. It’s a pity that many people forget this. It’s hard to count how many times I’ve heard proposals: “Let’s make the model on mock data.” Yet this is a meaningless activity; the only thing the model can learn is the rules we used to generate this data, not the actual business rules*.

The data we work on is customer data, so to “get” it, it is necessary to convince our client or partner that providing it is required to train the model, then to ensure the proper contracts under which the data will be transferred, and finally to reassure the client that the data will be safe with us. To make it as easy as possible for our clients to provide data, we help them select the minimum set of data needed to train the model, anonymize it and export it.

9. A significant risk is data leakage. How to prevent it? How to protect yourself and your client?

In this case, data leakage can be understood in two ways; overall, both carry much risk. Probably most people, when hearing about data leakage, will think about getting out of the company the information we use to train models. Such an event would, of course, be a very serious incident, primarily since we usually work on financial data. That’s why we have a set of procedures that we stick to when working with data. In addition, we mitigate the risk through joint workshops with clients, where we establish the minimum amount of data needed to train a model. 

Machine learning, on the other hand, relates to a different type of data leakage. It happens when some information in a training dataset should not be there. Most often, this data will not be available to the model during production. For example, when we get a data warehouse export from a customer, but the model is ultimately supposed to work on a transactional system. The best way to prevent this is through technical workshops with the client, where we discuss in detail the data they send us. Cooperation with the team on the customer side is vital here. 

Another case of data leakage is when identical examples are found in the training and validation sets. Then, instead of generalizing the knowledge, the ML model learns the individual examples by heart. Probably, each of us once had an exam for which we were given a list of questions. Then, to provide the correct answer, you don’t need to understand the topic, but only to learn that “if there is a question with this figure, choose answer B”. An analogy can happen with a model that does a great job during validation but completely fails after deployment to production. Such cases are usually the result of errors in pre-processing, so you need to pay much attention to code verification. We also arrange internal workshops during team meetings at the company, demonstrating the steps of working with the most important data. 

10. You have extensive ML experience and talk about it with great passion. Also, conduct workshops and classes at universities. Do you like to share your knowledge?

Yes, I do like to share and talk about various issues. There was a time when I worked at AGH and at the same time taught programming classes for children and teenagers. Then I taught almost every day. Now, it’s more like a few – a dozen meetings a year, but it still gives me great satisfaction. I fondly remember the lecture at this year’s Student IT Festival. After the presentation, quite a few people stayed, and I still answered a series of questions for 45 minutes. 

11. Does the company support you in all these activities?  

Yes, it takes much time to prepare for a lecture or workshop. Fortunately, Ailleron invests heavily in employee development and company promotion through educational initiatives, so I have adequate time to prepare. The company, mainly through the hands of colleagues in Marketing and Employer Branding, strongly helps organize, edit materials and promote them. This was evident, for example, in the webinar about ML and chatbots that we conducted with BNP Paribas.  

Currently, we are also working on a series of articles about ML together with the whole team. This series should be published in the last quarter of this year. I encourage you to stay tuned and read it, of course. 

12. Have you had the opportunity to share your experience internally as well?

Yes, recently I’ve been doing most of the training for people inside the company. Lately, we had a very cool meeting with salespeople where we talked about the details of our models so that they can tell customers about them even better. 

13. As of right now, ML is doing really well. Many young people want to follow this path. What are the job options and opportunities for growth in the ML field at Ailleron?

Indeed. Today, ML is a very prospective field where you can work on exciting problems and develop quickly. On the other hand, I can’t think of any other area that is changing as fast as ML in the last years. You have to be ready for continuous development and learning.  

In Ailleron, there are many opportunities to start working in this area. We have organized internships in ML, among others. I have excellent team members who came to us just from internships. Moreover, there is also the possibility of re-branding. Some people in our team had other roles before and are now doing a great job as ML engineers.  

The topic of Machine Learning is vast and quite complicated, but how intriguing. All the more, Kuba, we thank you for bringing this fascinating subject closer to us in such a practical way and for sharing your impressive knowledge again. As you can see, this is a lightning-fast industry for professional development. It’s an opportunity for young people who set their sights on new technologies but also for people who want to try something new and exciting in their work. 

*It is possible to build models on artificially generated data – there are even whole start-ups engaged in generating such data, but still, the artificial data is generated based on real data to map the relationships present in it.