SCM Change Set History Replication and Migration with Rational Team Concert

Have you ever been asked to move your RTC development environment to another installation / deployment and wondered what will happen to all my existing SCM Change Set history? We know that Project Move and/or Copy has been a long requested feature on Jazz.net. So, I know I am only singling out SCM Change Set history as one of many items one would want to migrate as part of a RTC Project Move or Copy. Now, why did I have to even consider going through this?

I’m supporting an internal team that is adopting the Rational solution for Collaborative Lifecycle Management as part of their enterprise modernization effort. Some of the teams were already using RTC 3.0.1.x on an internal deployment site. But, the organization as a whole was not using a consistent set of tools which would help them achieve the benefits of collaborative lifecycle management(CLM). The organization decided to roll out CLM 4.0.6 on s separate internal deployment site and thus, begin to make use of the benefits of CLM. Teams with existing data in RTC 3.0.1.x (workitems, source control, requirements, attachments, etc) were going to need help migrating these artifacts to the new environment which is being set up with lifecycle projects. Workitem migration was accomplished using CSV files and RTC’s CSV export and import feature for CSV files. We even had an internal attachment migration utility that would help bring over the attachments of workitems which is not supported OOTB but can be done via the RTC API. So, enough background on the how I got to the SCM Change Set history migration question.

There is already an existing Jazz.net article that covers this to some extent, Flow changes cross repositories with Rational Team Concert. However, I quickly realized I didn’t need to do all of what the article discussed to accomplish my goal: Migrate change set history from one repository to another.

Procedure

The procedure I followed to complete this two phase change set history migration is as follows:

  1. Install a 3.0.1.x interim server running the default Tomcat and Derby deployment.
  2. Install a 3.0.1.x RTC eclipse client and connect to both source and target repositories.
  3. Migrate all the streams from source CCM server to target interim CCM server. Note: The fastest way to do this is to use stream to stream flow targets instead of repository workspaces. See Jazz.net library article 535 for details. I also would have liked to have tried the suggestion on how to migrate interim baselines but I ran into problem #2.
  4. Upgrade the 3.0.1.x interim server to 4.0.6
  5. Install a 4.0.6 RTC eclipse client and connect to both source and target repositories.
  6. Repeat step 3 to migrate all the streams on the interim source server to their final destination in the target server CCM project area.

Challenges

Some challenges did present themselves along the way which I will present below along with the workaround or solution.

Problem #1: Version Incompatibility of source and target servers

The first obstacle I ran into was the fact that each repository was at a different level and the article clearly mentioned that distributed SCM or what make change set replication cross repositories possible is not going to work from 3.0.1.x -> 4.0.6. See Appendix: Backwards Compatibility and Distributed SCM.

Workaround #1

Simply set up an interim server, migrate/replicate all the streams, manually, as outlined in the Jazz.net article, then upgrade the interim server to 4.0.6, and manually migrate/replicate all the streams to the target 4.0.6 server.

Problem #2 – Server to Server network connectivity

The next issue that I ran into was not being able to fully complete the replication of all the interim baselines during the first phase of the replication, 3.0.1.x to 3.0.1.x. Now, what does that have to do with server to server network connectivity? Well, RTC 3.0.1.x did not provide me much off a clue with its error logging. See workitem 315822. I proceeded to retrieve the baselines that I could which usually meant I could only retrieve the last baseline. It wasn’t until I went to complete the second phase that RTC client error logging gave me a better clue as to what was going on (with a little help from RTC SCM developers). I had a server network visibility issue. The interim server was behind a firewall and its public URI was also not known as part of the DNS of my network. Thus, the operations would fail with “unknown host” socket errors.

Workaround #2

What I was able to do may surprise you depending on how much you have read about how to properly set up the public URI of a CLM server. I was able to zip up the CLM install directory of my interim server installation and copy it to my laptop which is not behind a firewall. I then proceeded to unzip it and enter a local hosts file entry to map my IP address to the public URI hostname of the interim server.

Problem #3 – Insufficient JVM memory

When I had solved challenge #2, I proceeded with the second phase change set replication. The target server and the source server were now both at 4.0.6 level. However, when I tried to complete one of the first delivery or accepts, it would never complete. Change set replication goes through rounds to deliver the data. I followed the suggestions in the article regarding large replication sets. And still no complete delivery.

Workaround #3

A colleague suggested I increase the JVM memory of the source server. I changed the following default settings in the server.startup.bat first to 8G and then finally to 10G: -Xmx10G -Xms10G -Xmn1280M. This follows the suggestions to make the nursery 1/8 of the max and min values. Of course, I have enough memory to spare since my laptop has 16GB RAM. With these changes, the deliveries proceeded to complete.

Summary

In conclusion, it is possible to migrate SCMchange set history from a 3.0.1.x source server to a 4.0.x target server. It just takes a bit of patience and ingenuity. It is not necessary for both the servers to have a friend relationship. The only items needed are the following:

Special thanks to Ralph Schoon and Tim Feeney for their valuable suggestions.

 

 

Advertisements

Reporting view differences between RTC, RRC and RQM

Someone recently asked me this question: I am configuring Insight (or RRDI) with CLM 4.0.x.  When I open reports in RRC or RQM, I am taken to the report server URL. Is there a way to avoid this by importing custom reports into CLM? Can the reports be run from within CLM without being taken to the report server URL?

I get this question alot and I always have to spend some time remembering how this works.  The answer is that it depends on which product: RTC, RQM or RRC.  Thanks to my colleague, Geoff Rosenthal, for this explanation and screenshots.

RRC

The Reports view in the RRC Web UI can be invoked using the Welcome to Reports selection of the Reports menu.

20130830BlogPostPic1

  • RRC shares a similar Reports menu with the RTC and RQM web clients.
  • Unique to RRC is the Generate a Document-Style Report selection which invokes a Rational Reporting for Document Generation (RRDG) wizard.  This wizard only exists in RRC today.
  • The View Reports selection is only available **if** a custom reporting server has been connected to the RRC Jazz Team Server(https://<clm server>/jts/admin#action=com.ibm.team.reportsManagement.configureCustomReportsConnection)
    • The View Reports selection takes the user from the RRC Web client to the homepage for the connected report server.
    • This screenshot displays a RRDI report server. An Insight report server looks almost identical.

20130830BlogPostPic2

Note:  The report server requires credentials that are typically shared with CLM credentials (the Jazz Namespace).  Thus, a user who logs into CLM should also have access to the Report Server with the same credentials.  The user doesn’t have to re-enter their credentials to access the Report Server if the Jazz Namespace has been configured accordingly.  Note that a user can set their own desired homepage in the Report Server.

Currently, RRC does not provide a mechanism to view report server reports natively in the RRC Web UI.  Thus, to view reports, RRC users must navigate over to the Report server.What about viewing custom reports via a dashboard viewlet? It is possible to add a custom reports viewlet to a RRC dashboard which includes a report server template resource as defined in RTC or RQM.  Since RRC does not provide functionality to view Report server reports from within the RRC clients, these custom report viewlets must come from either the RTC or RQM widget catalogues.

RTC

Now, let’s look at the Welcome to Reports selection from the Reports menu of the RTC web client.

20130830BlogPostPic3

  • RTC shares a similar Reports menu with the RRC and RQM web clients
  • The View Reports selection will take the user to the RTC Shared Reports area for the current projects
  • The Create report from resource selection will take the user to the Report Resources area. The Report Resources area allows a RTC user to create a report resource (report template) from a report that was created with a report server using either Report Studio or Query Studio, RPE or other report template creation tools.
  • The Organize your Reports selection will take the user to the RTC My Reports area for the currently logged in user
  • The Perform additional reporting tasks selection is only displayed **if** a custom reporting server has been connected to your RTC Jazz team server (https://<clm server>/ccm/admin#action=com.ibm.team.reportsManagement.configureCustomReportsConnection)
    • The Perform additional reporting tasks selection will take the user from the RTC web client to the homepage on the connected report server, such as:20130830BlogPostPic4
    • This screenshot above is of a RRDI report server.  An Insight report server looks almost identical.
    • Note:  The report server requires credentials that are typically shared with CLM credentials (the Jazz Namespace).  Thus, a user who logs into CLM should also have access to the Report Server with the same credentials.  The user doesn’t have to re-enter their credentials to access the Report Server if the Jazz Namespace has been configured accordingly.  Note that a user can set their own desired homepage in the Report Server.

RQM

Lastly, let’s take a look at the Reports menu of the RQM Web client.

20130830BlogPostPic5

  • RQM does NOT have a Welcome to Reports selection
  • RQM shares a similar Reports menu with the RTC and RRC web clients
  • The Shared Reports selection will take the user to the RQM Shared Reports area.  This allows a RQM user to create a report resource (i.e. a report template) from a report that was created with the Report Server (either Query Studio or Report Studio), Rational Publishing Engine, or other report template creation tools.
  • The My Reports selection will take the user to the RQM My Reports area for the current user
  • The Perform additional reporting tasks selection is only displayed **if** a custom reporting server has been connected to your RQM Jazz team server (https://<clm server>/qm/admin#action=com.ibm.team.reportsManagement.configureCustomReportsConnection)
    • The Perform additional reporting tasks menu selection will take the user from the RQM client web page to their homepage for the connected Report Server, such as:

20130830BlogPostPic6

  • This is a screenshot of the Rational Reporting for Development Intelligence (RRDI) report server.  The Rational Insight report server looks almost identical to this screenshot.
  • Note that the Report Server requires credentials that are typically shared with CLM credentials (the Jazz Namespace).  Thus, a user who logs into CLM should also have access to the Report Server with the same credentials.  The user doesn’t have to re-enter their credentials to access the Report Server if the Jazz Namespace has been configured accordingly.  Note that a user can set their own desired homepage in the Report Server.

Getting started with a Rational Team Concert (CLM) deployment

I recently had to put together a quick “how-to” get started list of material regarding deploying Rational Team Concert v4.0.2 in a single server department topology with WAS and DB2.   In addition, there was a request to include a reverse proxy and IHS plugin to the topology.  This list applies to any CLM product: RTC, RRC, or RQM.

I wanted to share the list of references I compiled.

CLM 2012 Deployment Guide <–Overall guide that touches at a high level the points mentioned below.

1) First and foremost, I would read the Planning to deploy and Install topic in our infocenter: http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/c_planning_install.html

It includes information on topology planning including reverse proxies: http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/c_reverse_proxy.html

2) Regarding WAS, here is the parent topic: http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/c_deploying_was.html

Notice the topic on configuring a reverse proxies using IHS: http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/t_config_reverse_proxy_ihs.html

The above topic includes the links to two Jazz.net articles written as a how-to for reverse proxy configuration. They were written for 3.0.1 but they still apply for 4.0.x.

Configuring Enterprise CLM Reverse Proxies, Part 1: Understanding Reverse Proxy

Configuring Enterprise CLM Reverse Proxies, Part 2: WebSphere and IHS Plugin method

Notice the topic about the Jython script vs Admin console approach for deployment of CLM on WAS.

The entire section on deploying on WAS should be read as it also discusses single Signon.

This article discusses deploying on WAS+IHS when JTS and CCM are on separate app server profiles: https://jazz.net/library/article/1192

3) There is a patch needed when using  BIRT reports on dashboards and WAS 8.0.0.x, see workitem 261100

4) Regarding DB2, here is the link to the Setting up the databases topic for DB2: http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/t_s_server_installation_setup_db2.html

I usually follow the steps here before deploying on the app server and subsequently running the setup wizard.

5) The infocenter now provides a feature called an Interactive Install Guide where you can specify the details of your environment and it will generate a step by step guide. This is good to read after first glance at all the individual topics because it pulls it all together for you in sequential format. http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/roadmap_form.html

6) In case you did not already know, you cannot change the public URI of CLM applications, at least not without a server rename as of v4.0.x, so, please read the information regarding how to choose an appropriate public URI.  http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0m2/topic/com.ibm.jazz.install.doc/topics/c_planning_URLs.html

7) For your reference, here is the System Requirements article for CLM 4.0.2: https://jazz.net/library/article/1109 CLM 4.0.3 updates are in draft mode here: https://jazz.net/wiki/bin/view/Deployment/CLMSystemRequirements

8) I perused the Jazz.net library for articles related to deployment. Here is a list of ones are I found useful.

http://www.ibm.com/developerworks/rational/library/create-server-side-certificates-collaborative-lifecycle-management/index.html?ca=drs-

https://jazz.net/library/article/1258

https://jazz.net/library/article/1143

9) CLM Administration workshop https://jazz.net/library/article/831/ This workshop is a MUST for anyone planning and implementing a deployment of any CLM application.  In this workshop, we take the student from an eval topology to a department or enterprise topology, and demonstrate how to install/configure a reverse proxy and move to WAS+DB2+LDAP. This material was developed using CLM 4.0.

10) For questions regarding backups, here is an article to have around: https://jazz.net/library/article/795

11) TIP: Whenever you are re-directed to an infocenter link such as ones that start with pic.dhe.ibm.com/infocenter/clmhelp, you can always make sure you are reading the latest, if you substitute v4r0m2 instead of v4r0 or v4r0m1.

12) Tuning the Rational Team Concert 4.0 server

13) Collaborative Lifecycle Management 2012 Sizing Report (Standard Topology E1)

14) Youtube playlist for Install: http://www.youtube.com/playlist?list=PLZGO0qYNSD4W3XDwY59GybtiwVd01YRe0

15) Guide to better performance for Jazz applications is a blog post by Dan Toczala that captures many performance related best practices for CLM deployments.