Steering committee modified protocols
The sleight of hand that AFAS needed
Door Alessandro Tessari
By Alessandro Tessari
January 18, 2021. Nine people from all over the university have logged on for a special online meeting. On the other side of the screen is Erwin Boelens, programme manager of Best Practice 2020.
‘So the original acceptance tests were not done according to the original protocols?’ asks Nasser Kalantar, professor of nuclear physics at the Faculty of Science and Engineering (FSE).
‘Yes’, agrees Boelens.
Kalantar pushes on. ‘Normal procedure for such a project would be factory acceptance tests by AFAS, then site acceptance tests’, he says. ‘Were these protocols bypassed or adjusted? And if so, with what justification?’
‘Acceptance tests were performed on a subset of the data. Not all’, Boelens replies. ‘Protocols were not skipped, but modified.’
Rules of conduct
Boelens’ answers shocked the members of the special committee attending the meeting, who were charged by the university council to find out what had gone wrong with the introduction of AFAS, almost a year earlier on January 1, 2020.
‘It made us wonder whether the whole process had been fair’, says David Jan Meijer, who was part of the panel as member of the university council and chair of student party De Vrije Student.
What Boelens told us was the equivalent of scientists tampering with data
‘To me, what Boelens told us was the equivalent of scientists tampering with data to get the results they want’, says Mariano Mendez, professor in astronomy and astrophysics at FSE. ‘I would imagine there are rules of conduct for administrative practice, just as there are for research.’
However, the committee never reports its findings. Due to some members being on sick leave, its meetings end after only a few more gatherings. The problems the UG faced with the system are never explained.
One thing is absolutely clear: the original plan of Best Practice 2020 – as the operation was called – required at least two months of testing the software by UG staff on UG data. And that wasn’t possible. Only partial testing could be completed, and that’s when protocols were modified.
Concerns
Preparations for the implementation of AFAS had been underway for some time. The university had good reason to want an integrated software system to replace a number of old ones. And there was a deadline: the contract with the payroll system would expire on January 1, 2020.
AFAS is a very nice centralised system, but the UG is deeply decentralised
But there were concerns, too. For one, the UG was the first university customer for AFAS. But the structure and culture of a university is different from the organisations the system was designed for.
‘AFAS is a very nice centralised system’, explains Meijer. ‘But unfortunately, the UG is deeply decentralised. Each faculty runs in a different way.’
Jacob Jolij, assistant-professor of theoretical and philosophical psychology at the Faculty of Behavioural and Social Sciences, agrees: ‘We spent decades decentralising the control of tasks to the single faculties, and now all of a sudden we implement a totally centralised system.’
Unbalanced
That would make listening to people who were actually supposed to use the software quite important. But that too, happened only to an extent. FSE technician Arjen Kamp was part of a group that was supposed to evaluate the user end of the purchasing part of AFAS. ‘There were around ten people in my group’, he recounts. ‘But I was the only one from a faculty. The others were from finance or the purchasing department. It was totally unbalanced.’
The group had only one single meeting, but that was enough for Kamp to realise fundamental pieces were missing. ‘They had a complete misconception of what the faculties needed’, Kamp says.
Others in his faculty agreed wholeheartedly. FSE made it clear to the university board several times that it was not ready to go with the new system, but that plea landed on deaf ears.
Hiccups
October, 2019. According to the schedule, UG staff were supposed to start testing the new system on real data this month. Data that would be copied into the new AFAS environment from the previous one.
However, at the end of September, the data was still not available due to technical issues. ‘We couldn’t stick to the original schedule’, Boelens says. ‘We had quite a few hiccups during the conversion of data.’
They had a complete misconception of what the faculties needed
The full conversion of the data wasn’t accomplished until November 25, just one week before the UG board was due to make a go/no-go decision. And that meant that a key part of the planned protocols could not be followed.
Boelens and his team went on with what they had. Acceptance tests were run on a subset of data and on functionalities that did not require real data. Issuing sick leave for example, or placing orders. That seemed to be going well.
Judgement call
At the moment of the go/no-go decision, the AFAS steering group made a judgement call and decided that, despite the fact that the original requirements were not met, what they had done was enough.
‘The original plan suffered an alteration, but in everyone’s opinion it wasn’t enough not to proceed with the implementation’, Boelens says. ‘We recognised that all major and necessary things were in place, like the payroll part. The interface was working and most of the things were configured and ready to go.’
The advice to the board of the university was clear: proceed. And they did. What followed was months of administrative chaos that still hasn’t been completely resolved.
Boelens still stands firmly behind the university’s decision. Postponing, he argues, would not have led to any more confidence or security regarding the problems that lay ahead. ‘The proof is in the pudding, and the pudding is when you go live with the system, especially with users,’ he says.
Pandemic
There would be problems, everyone knew that. Boelens too. ‘Especially in the first quarter, testing or not. With such huge system changes, inconvenience and troubles always have to be taken into account.’
He had already planned ways to support and train users. But then the pandemic hit and worsened an already difficult situation. ‘We could not get in touch with users in the way that we wanted.’
With such huge system changes, troubles always have to be taken into account
But others feel differently. ‘Several tests didn’t meet the requirements before the deadline’, says committee member Ben Karsdorp with the Centre for Information Technology. ‘Normally that should mean no-go, but in this case, it was still a go. That is weird, because you are ignoring the basic principle of a go/no-go moment.’
‘Some tests were surely done, but apparently not all the necessary tests’, says Kalantar.
Both wonder whether the no-go decision was ever a real option and whether there was an actual back-up plan.
Partial launch
Karsdorp would like to know why the university didn’t go for a partial launch. AFAS has different modules, so why not just launch the payroll part and postpone the rest? Or extend the contract with the former payroll system? The university of Eindhoven was also supposed to implement AFAS in 2020, but postponed it for a full year to avoid chaos. Why not do that?
Boelens says that, of course, there was a back-up plan.‘The plan in case of a no-go decision was to continue using the old systems and postpone implementation for a year’, he says. But that simply wasn’t necessary. ‘We have met the truly necessary requirements and paid attention to all the crucial parts. I guess it is a matter of opinion where you draw the line on a go/no-go decision. We acknowledge there were problems and we have to learn from our experiences for future implementations.’
A partial launch – as suggested by Karsdorp – was not possible. ‘In our case, AFAS is one complete system. Its modules are strictly connected, and at the time of the go/no-go decision, it was too late to divide them up.’
New functionality
Since the structure and culture of a university is different from other organisations, not everything could be tailored towards the university’s needs, he says. ‘But we have seen AFAS develop and they still are developing new functionality in order to fit the requirements of the university better.’
Jolij does not blame the AFAS system, but questions the choices that were made. ‘AFAS is a good product. A successful implementation would have been possible only if they would have spent one year talking to and testing into faculties to harmonise AFAS with all the internal fundamental differences.’
Meijer agrees. ‘Certain conditions had to be met before the system could be introduced. But the board of the university did not follow those and UG did not look at their own requirements.’