A probabilistic transformer for predictive monitoring of editorial process

Objectives. The aim of this study is to develop an approach for predicting the timing of several key stages in multi-stage processes.

Methods. A method is proposed based on processing event sequences taking into account stages, time intervals, and information about process participants, as well as a consistent consideration of the relationships between the predicted stages.

Results. Experiments conducted on editorial document flow data demonstrated improved accuracy compared to recurrent neural networks and a significant advantage in the early stages compared to gradient boosting models.

Conclusion. The method allows obtaining of consistent interval forecasts for several stages of a process and can be used to quickly update the estimates without the use of specialized equipment.

1.

2. Dumas M., La Rosa M., Mendling J., Reijers H. A. Fundamentals of Business Process Management, 2nd ed. Berlin, Springer, 2018, 527 р. https://doi.org/10.1007/978-3-662-56509-4.

3. Aalst W. M. P. van der. Process Mining: Data Science in Action, 2nd ed. Berlin, Springer, 2016, 467 р. https://doi.org/10.1007/978-3-662-49851-4.

4. Verenich I., Dumas M., La Rosa M., Maggi F. M., Teinemaa I. Survey and cross-benchmark comparison of remaining time prediction methods in business process monitoring. ACM Transactions on Intelligent Systems and Technology, 2019, vol. 10, iss. 4, рр. 1–34. https://doi.org/10.1145/3331449.

5. Teinemaa I., Dumas M., La Rosa M., Maggi F. M. Outcome-oriented predictive process monitoring: review and benchmark. ACM Transactions on Knowledge Discovery from Data (TKDD), 2019, vol. 13, iss. 2, рр. 1–57. https://doi.org/10.1145/3301300.

6. Maggi F. M., Di Francescomarino C., Dumas M., Ghidini C. Predictive monitoring of business processes. Advanced Information Systems Engineering: 26th International Conference, CAiSE 2014, Thessaloniki, Greece, 16–20 June 2014, рр. 457–472 (Lecture Notes in Computer Science; vol. 8484). https://doi.org/10.1007/978-3-319-07881-6_31.

7. Teinemaa I., Dumas M., Maggi F. M., Di Francescomarino C. Predictive business process monitoring with structured and unstructured data. Business Process Management: 14th International Conference, BPM 2016, Rio de Janeiro, Brazil, 18–22 September 2016, pр. 401–417 (Lecture Notes in Computer Science; vol. 9850). https://doi.org/10.1007/978-3-319-45348-4_23.

8. Kratsch W., Manderscheid J., Röglinger M., Seyfried J. Machine learning in business process monitoring: A comparison of deep learning and classical approaches used for outcome prediction. Business & Information Systems Engineering, 2021, vol. 63, iss. 3, рp. 261–276. https://doi.org/10.1007/s12599-020-00645-0.

9. Evermann J., Rehse J.-R., Fettke P. Predicting process behaviour using deep learning. Decision Support Systems, 2017, vol. 100, рp. 129–140. https://doi.org/10.1016/j.dss.2017.04.003.

10. Camargo M., Dumas M., González-Rojas O. Learning accurate LSTM models of business processes. Business Process Management: 17th International Conference, BPM 2019, Vienna, Austria, 1–6 September 2019, рp. 286–302 (Lecture Notes in Computer Science; vol. 11675). https://doi.org/10.1007/978-3-030-26619-6_19.

11. Tax N., Verenich I., La Rosa M., Dumas M. Predictive business process monitoring with LSTM neural networks. Advanced Information Systems Engineering: 29th International Conference, CAiSE 2017, Essen, Germany, 12–16 June 2017, рp. 477–492 (Lecture Notes in Computer Science; vol. 10253). https://doi.org/10.1007/978-3-319-59536-8_30.

12. Navarin N., Vincenzi B., Polato M., Sperduti A. LSTM networks for data-aware remaining time prediction of business process instances. 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA, 27 November – 01 December 2017. Available at: https://arxiv.org/pdf/1711.03822 (accessed 10.03.2026). https://doi.org/10.1109/SSCI.2017.8285184.

13. Polato M., Sperduti A., Burattin A., de Leoni M. Time and activity sequence prediction of business process instances. Computing, 2018, vol. 100, iss. 9, рp. 1005–1031. https://doi.org/10.1007/s00607-018-0593-x.

14. Senderovich A., Di Francescomarino C., Ghidini C., Jorbina K., Maggi F. M. Intra and inter-case features in predictive process monitoring: A tale of two dimensions. Business Process Management: 15th International Conference, BPM 2017, Barcelona, Spain, 10–15 September 2017, рp. 306–323 (Lecture Notes in Computer Science; vol. 10445). https://doi.org/10.1007/978-3-319-65000-5_18.

15. Kazemi S. M., Goel R., Eghbali S., Ramanan J., Sahota J., …, Brubaker M. Time2Vec: Learning a vector representation of time, 2019. Available at: https://arxiv.org/pdf/1907.05321 (accessed 10.03.2026). https://doi.org/10.48550/arXiv.1907.05321.

16. Lim B., Arık S. Ö., Loeff N., Pfister T. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 2021, vol. 37, iss. 4, рр. 1748–1764. https://doi.org/10.1016/j.ijforecast.2021.03.012.

17. Shazeer N., Mirhoseini A., Maziarz K., Davis A., Le Q., Hinton G., Dean J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017. Available at: https://arxiv.org/pdf/1701.06538 (accessed 10.03.2026). https://doi.org/10.48550/arXiv.1701.06538.

18. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., …, Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 2017, vol. 30, рр. 6000–6010. https://doi.org/10.48550/arXiv.1706.03762.

19. Gneiting T., Raftery A. E. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 2007, vol. 102, no. 477, рp. 359–378. https://doi.org/10.1198/016214506000001437.

20. Gorishniy Y., Rubachev I., Khrulkov V., Babenko A. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems (NeurIPS), 2021, vol. 34, рр. 18932–18943. https://doi.org/10.48550/arXiv.2106.11959.