AppDynamics upgrade / Glassfish Master password / Keystore issues?

We were having real problems upgrading AppDynamics from 4.2 to 4.3 then on to 4.5. All of the issues where caused by things that were not mentioned in their documentation.

Hopefully this information might prove to be useful to someone else in the same situation.

Our main issue was been caused because we had a custom password securing our Glassfish Master / Keystore & Keystore Keys, all of which was the same, as they need to be, because of this the installer was failing as it doesn’t ask you to insert your password as a parameter during installation, you need to change it back to the default of ‘changeit‘ before commencing an upgrade when we tried to get asadmin to do this for us, we got the error:

asadmin change-master-password --savemasterpassword=true
Enter the current master password>
Enter the new master password>
Enter the new master password again>
Keystore was tampered with, or password was incorrect
Command change-master-password failed.

After many hours we worked out that you first needed to:
Browse to the folder: \AppDynamics\Controller\appserver\glassfish\domains\domain1\config and type the following commands to set the password back to the default one.

keytool -storepasswd -keystore keystore.jks
Enter keystore password:
New keystore password: changeit
Re-enter new keystore password: changeit
keytool -keypasswd -alias glassfish-instance -keystore keystore.jks
Enter keystore password: changeit
Enter key password for :
New key password for : changeit
Re-enter new key password for : changeit
keytool -keypasswd -alias reporting-instance -keystore keystore.jks
Enter keystore password: changeit
Enter key password for :
New key password for : changeit
Re-enter new key password for : changeit
keytool -keypasswd -alias s1as -keystore keystore.jks
Enter keystore password: changeit
Enter key password for :
New key password for : changeit
Re-enter new key password for : changeit
The most important step, which is missed in the documentation, if you’ve secured this file using anything but ‘changeit‘ then it’ll cause the above asadmin command to fail everytime, even half way through an upgrade.
keytool -storepasswd -keystore cacerts.jks
Enter keystore password:
New keystore password: changeit
Re-enter new keystore password: changeit

Other possible issues which you might encounter:

JRE Path is incorrect

Error: could not open `\AppDynamics\Controller\jre\lib\amd64\jvm.cfg’ The above error is telling you that your JRE path is messed up and it cannot locate the required files.

Solution:

  • check location of above file
  • Edit \AppDynamics\Controller\appserver\glassfish\config\asenv.bat
    • Make sure ‘set AS_JAVA=’ is point to the root of the JRE with the above file.

Error:

Stage [Discover Controller SSL certificate] failed due to
[Task failed: Discovering SSL certificate
on host: HOST
as user: HOST$
with message: Keystore was tampered with, or password was incorrect]

Solution: When upgrade the controller from 4.5 to 4.5, you can set your homemade password in for Glassfish / Keystore / Keys in controller.groovy

keypasswd = "changeit"
storepasswd = "changeit"

but this still failed until we had set all our passwords back to ‘changeit‘ see above on how to do this, then the above controller.groovy file to ‘changeit‘ as well.

Error:

Task failed: Starting Reporting Service
on host: HOST
as user: HOST$
with message: Expected is [RUNNING], but actual result is [STOPPED].

Solution:

Stop the AppDynamics Reporting Service and restart it, then continue the installation back in the AppDynamics 4.5 Enterprise Console.

Good luck out there, upgrading AppDynamics is not an easy task!

We are now looking to move to SaaS because it caused us so many issues.

One comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s