February 2022 on vAMSYS

[Lukas Jankauskas]

As you might have seen - vAMSYS logo has changed and there are no quirky loading messages on Select page and within Orwell, where the new design has been implemented. 

Neither Team vAMSYS, nor I am immune to feelings or emotions - we are not robots - I believe our temporary logo makes it quite clear where and with who do we stand. 

I look in awe at the determination, resilience and sacrifices of the Ukrainian People and their military to protect their country - be it alone and on their own. We Stand firmly in Solidarity and sorrow with the People of Ukraine as shocking events continue to unfold. (Discord #announcements - 25th February 2022)

 

----------

End of February update! Are you excited? I am, as I get to write this one and George may or may not hold my hand writing it. 
So let's get cracking.

 

Server Migration

23rd of February was our migration day. This was the 3rd time in vAMSYS history where we significantly overhauled our infrastructure. As the rarity of such event implies - this was a massive change and it involved half-a-day worth of downtime. I would like to thank all our friends for actively disseminating the migration information, not making a criminal case out of scheduled downtime and working with their pilots to make sure they are aware of what is going on so neither of us get bogged down with support requests - your participation and understanding made it the easiest and least distracting of migrations we have done so far, and the first migration where I kept a running commentary/log of what we were doing in #announcements chat of our Discord - hope it was useful and kept you in the loop.

 

We have mentioned it before, but I'll put it here again - we consolidated multiple bare-metal servers leased from OVH (our service provider) into 2 much more powerful machines - power of which you have already noticed and told us about. We are delighted to hear that it had immediate positive impact. This migration has set us up for at least a year if not more.

 

Our new infrastructure is made up of 2 identical servers with following specs each: AMD Epyc 7642 CPU (48 Cores, 96 Threads; Base clock of 2.3 GHz, tubo up to 3.3 GHz), 256 GB ECC 2933MHz RAM; 2x480 GB SATA SSDs and 2x1.92TB NVMe SSDs. Servers and their resources are split into Virtual Machines (VMs) of various importance to our operations and CPUs/RAM/Space as needed. We hold 40 IPv4 addresses - some in reserve for our future needs, others actively in use for our virtual machines responsible for every facet of vAMSYS functionality. 

 

Allow me to briefly summarise our current structure of VMs:

• 1 Primary Database (PSQL) VM accepting write requests (inserts into database - think of new bookings, new pilots) and limited read requests;
• 2 Secondary Databases (PSQL) VMs serving 95% of all read requests of the database (as the name implies - reading all the data in our database - pretty much everything you see);
• 2 vAMSYS.io web VMs - they are responsible for rendering the php and pulling the data for you to see and make bookings on;
• 1 MYSQL database VM - used exclusively by Invision Power Board forums, known as vAMSYS Community Forums;
• 1 Redis database VM - our volatile storage storing temporary information, like the minute-by-minute flight lists and data for the maps;
• 1 Neo4J database VM - our graph database responsible for providing data for maps to draw a line airways and waypoints and validating routes inserted by VAs;
• 1 Load Balancer VM - responsible for splitting requests between databases and web servers equally and fairly. It also secures our various exposed endpoints to https;
• 1 Forum VM - self explanatory;
• 1 Weather VM - again - should be obvious - it provides and keeps historical record of ATISes and TAFs;
• 1 Mapper VM - responsible for providing map images on booking, destination, PIREP review, event and other pages - basically - every map you see is rendered by our in-house server;
• 1 MNA VM - yes, MNA still receives support from vAMSYS;
• 1 DirectAdmin VM - provides emails for Team vAMSYS, hosts our 'corporate' website, as well as homepages for some VAs;
• 1 Worker VM - one of the most-critical virtual machines in our use which is responsible for all scheduled and as-needed tasks, like PIREP processing.

We spent a month talking about it, a month preparing for it and a week implementing it, but our infrastructure migration has been successful - at the time of writing this post - entire migration has been complete and, other than the scheduled downtime where we transferred PSQL database only, none of you were any wiser of all the other things going on and happening seamlessly. Today, the last 3 remaining old servers have been put to pasture and re-rented to other companies and hobbyists. 

Platform changes over February

February has also been very busy and productive as far as our code refactor and style updates went. There were 240 code change commits to our codebase (8.5 a day). I have kept updating everyone in #v3-changes chat in Discord, but I'll summarise it here for posterity:

• New Select VA page - faster and in-line with our new design;
• Updates to Team vAMSYS internal pages (called Aeolus, from Greek mythology, a divine keeper of the winds and king of the mythical, floating island of Aiolia) - with code improvements and new design;
• Minor change to route importer, where route is now optional;
• Announcements Orwell sub-menu receiving new vAMSYS styling and corresponding code improvements;
• Pilots Orwell sub-menu receiving new vAMSYS styling, corresponding code improvements and rich datatables enabling you to more easily find what you need;
• Major changes to Orwell Data sub-menu where Importers and Exporters live, notably - the introduction of validator and a complete rewrite of importer/exporter logic. We are yet to have a stuck importer due to improperly formatted data;
• Temporary solution for allowing VAs to utilise SimBrief Aircraft IDs when their pilots dispatch via SimBrief;
• PIREP List, Livery List and Livery review received updates as well;
• Significant change how and when destination maps are generated;
• Live-chat addition to new Orwell pages;

There were also a few platform fixes and improvements not covered:

• Zendesk user sync fixes;
• IVAO network data parser fixes and resilience improvements;
• Fixes to logos in emails;
• Fixes to Engine Scorers (Number2FirstThen3Dual and Number4FirstThen3Dual);
• Improvements to VATSIM pre-file;
• Improvements to how and what data is sent to SimBrief in Dispatch;
• Individual VA login pages now bypass Select page when logging in with email;
• Fixes to PIREPs marked as needing reply;
• Fixes to transfer/repositioning flights now showing up correctly in Book Flight page;
• Addition of hide sidebar button.

 

Short-term changes to vAMYS 

I think it's fair to say 50% of Orwell is done - we will continue working on Orwell in the following order - Management Sub-Options followed by Events, followed by PIREP Review and finishing with Pages. 

Once we are done with Orwell, we will proceed with VDS.

We made meaningful progress this month - mostly because George was able to make the time and work on migration and I took 2 weeks of holidays from work, which allowed me to dedicate those weeks solely to vAMSYS. As much as I would like to - I cannot guarantee the same level of progress in March - we are, however - just as, if not more, committed and eager to improve the system.

New vAMSYS Features

General Outlook

As our upgrading/updating efforts continue, we cannot and will not make promises on new features - in fact, they are currently deprioritised. There are a few reasons for this - the three bits of vAMSYS (Phoenix, Orwell and VDS) are heavily interlinked - I cannot add a new setting in Orwell which will affect something happening in Phoenix; similar with VDS. Only when all the three systems are updated to the 'new' way, can we entertain the possibility of making real and significant functionality changes you have put forward on our Ideas platform. 

I want to set the record perfectly clear - our current efforts are solely focused on refactoring the code and improving usability - be it with clearer options, faster pages or less quirky/incoherent behaviour. We will add, where we can, self-contained features and your submitted ideas where they do not interact with other non-updated bits of vAMSYS. Once all this done, we will be in position to add the features we all agree vAMSYS needs to gain.

That does not, by any means, mean your suggestions are thrown out, ignored or forgotten. Ideas platform is there for this exact purpose - a place for you to make suggestions for improvement which we can look into in an organised fashion - as opposed to disjointed discord messages or emails.

POSCON

There has been a recent influx in feature requests for POSCON network connectivity implementation from our users - we are aware similar requests have been made to other companies in the virtual aviation world as well. vAMSYS supports any online network, and we are happy to allow users to enter network IDs for VA staff to monitor and report to network administrative staff. We implement functionality based mainly on time to implement and the impact we think it will have on users, with features that impact upon many users ranked as higher priority for implementation than those that impact fewer users.

We recently implemented the ability for users to specify their POSCON network ID in the settings. This was a relatively simple change to make with minimal development time. There are, so far, 11 users with a POSCON ID entered, across the entire platform.

It is therefore not currently in the best interest of the largest majority of users (or even customers - i.e. VA owners) to implement network connectivity scoring for this network, as this takes hours of development time to implement and, even assuming that only 1% of POSCON users that use our platform have entered an ID (which is a generous estimate), would only reach around 1,000 users. Our platform currently has over 30,000 registered users, and the actual number of POSCON users is likely much smaller.

We will continue to evaluate the priority of this request and will provide any updates as and when we are ready to provide them, but unless we make an announcement, it is highly likely there have been no changes. 

Platform Changes

We have begun reevaluating on how we support you - our customers. We can spend all the time in the world making changes and tweaks, but it would be pointless if we did not listen to your needs and your feedback.

Historically, Discord has been our first line of immediate support if we are online, with Zendesk (email tickets) as fallback and fill-in for things we cannot solve right away or are offline.

 As you might have gleaned from end of year update - vAMSYS is not rich, nor does have a lot of income to throw around - with that in mind, we are seriously considering if £60 per month for Zendesk, which is a glorified inbox, is even affordable to vAMSYS LTD, taking into account the new server costs, let alone delivering value for the price.

We have started experimenting with a chat-based system, which, so far, has received a very positive response from you. Having 'staffed' the livechat for a couple of days - I can say I am happy too and was able to connect with some Staff members who are not on Discord - it is fair to say that this option will remain and will displace Discord as preferred first line of support in due time.

We must, however, work out how we can tie-in the chat-based system to  help us keep track of issues which are not instantly solvable. We got a few ideas and we are exploring them in the month on March.

Last, but not least - we listened to your feedback about Ideas platform and how ideas should be prioritised and voted on. Whilst we do not have anything concrete yet - it is on our minds.

----------

Couple of messages before I close off this monthly update:

Do join me in saying congratulations to Anna.

And consider supporting George in 27 27 run to raise funds for Mind charity - more on it here https://discord.com/channels/398186537952477196/398187082696228875/947981324369666108

Till next month! 

Lukas

[Jack]

I think the question on everyone's mind is this... was @George Peppardactually holding your hand?

[Lukas Jankauskas]

On 03/03/2022 at 20:46, Jack said:

I think the question on everyone's mind is this... was @George Peppardactually holding your hand?

😄 😄 No, he wasn't. He did proof-read and made some changes after all

[George Peppard]

On 03/03/2022 at 20:46, Jack said:

was @George Peppardactually holding your hand?

He doesn't pay me enough!