Anyone familiar with ConnectWise Automate would probably not describe it as fast. It is powerful, robust, and slow as a turtle to load the application. The problems with ConnectWise Automate performance are particularly painful at large scales, and it seems to be almost a universal experience among partners. The root cause of these issues comes from how the application interacts with it’s MySQL database.
The Connectwise Automate performance problem:
MySQL is extremely configurable, and it needs adjusting to the particular use case it’s deployed for. Out of the box, the configuration is conservative and keyed for average use cases (like running a website). To get proper ConnectWise Automate performance MySQL will need to be configured appropriately.
The Details:
There are 458 variables in a stock install of MySQL that control how the database works under the hood (small regional jets have about 350 different controls for reference). ConnectWise provides some tuning guidance in their server install instructions (more on that later), but if you contact support regarding server performance issues they will likely tell you to add more resources (especially RAM) or to move to faster storage.
Obviously, a server needs adequate resources to operate correctly. However, blindly throwing resources at MySQL won’t help. Think about stock car racing. The engineers don’t simply put the largest engine possible in the car — there are other aspects of the design of the car that they modify in step with the engine that produce the finished product.
MySQL is a proper gentleman in terms of resource consumption. The default out of the box configuration assumes that the server is operating on mechanical drives. Even if the server is on the fastest solid storage known to man and has enough RAM to fit the entire database into memory it will still plod along as if it were on a SAS drive from 2004. Unless you explicitly configure MySQL to use those additional resources it won’t take full advantage of them.
Now, back to the vendor-supplied tuning instructions. MySQL creates a new thread for each connection to the server. Those threads all share certain caches, queues, and other structures inside of MySQL. Certain variables were designed to be adjusted in tandem, and when you look at the ConnectWise provided settings it’s almost like looking at a funhouse mirror. The max_connections variable default value is 151. The recommended value (which is legitimate) is 3000 (that’s almost a 2000% increase). Other elements of the database (buffers, caches, behavioral settings, etc.) aren’t scaled correctly in the ConnectWise guide, and they will need to be adjusted similarly for MySQL to perform well trying to satisfy the larger number of requests.
Now, back to our stock car analogy. The performance problem is quite clear if it’s pictured as a car with an engine that’s 2000% bigger than factory settings installed on the same suspension — it’s powerful, but it won’t operate correctly since it’s not supported properly. On paper, it might have impressive specifications, but anyone who looks at it will see that the car isn’t going to move efficiently anytime soon.
The solution: fixing slow Connectwise Automate issues
Here at Automation Theory, we have a couple of offerings that will help remedy the performance issues.
First up is our featured integration, the Database Tuner plugin (compatible with hosted and on-premise servers). It takes the max_connections variable and live performance data as a starting point and dynamically adjusts the other buffer, thread, and IO settings to scale resources properly in real-time (with no need to restart the database). This takes care of all the low hanging fruit with ease, instructs MySQL to use it’s resources to the fullest, and let’s Automate administrators get back to automating things. For anyone looking to improve the performance of their server (or for anyone experiencing control center lockups, monitors disabling, or long load times) we recommend starting with Database Tuner. Each Automate environment is different, and we start everyone off with a free 30-day trial; we believe the results will speak for themselves for anyone who tries it.
The companion product to the Database Tuner plugin is our Database Tune-up service (for on-premise servers only). This is a flat rate service where one of our automation theorists will analyze the particulars of a server environment and create a customized configuration file for the MySQL server variables that can’t be modified while the server is running. This service is dual-homed for performance and MySQL best practices. We advise this service for any partner who has not had their MySQL configuration file optimized for best practice, along with partners who have made significant changes to their server resources.
