D365 Finance & Operations and Dynamics AX Forum

Expand all | Collapse all

Automated regression testing for D365

  • 1.  Automated regression testing for D365

    TOP CONTRIBUTOR
    Posted Apr 24, 2019 12:04 AM
    I wonder if anyone here has implemented automated testing tool for regression testing.

    Are people using Microsoft provide tool? (are there any?) or 3rd party tools?
    Are you testing just UI or also interfaces?

    With the runaway train which is the "one version" upgrades every 6 months, how do you plan to deal with the amount of regression necessary?

    ------------------------------
    Zvika Rimalt
    Functional Consultant
    Vancouver BC
    ------------------------------


  • 2.  RE: Automated regression testing for D365

    Posted Apr 25, 2019 03:41 AM
    Zvika

    There are tools from MS that help you test your D365 implementation/running production environment. The tool is called Regression Suite Automation Tool, short RSAT.
    RSAT works together with LCS and Azure DevOps. In LCS BPM you store your task recordings coming from D365. In DevOps you manage your test plan and test cases.

    We are currently setting up RSAT as the test automation tool of choice. We'd like to use as much MS tools as possible so we stay in "their universe".

    During the RSAT workshop MS promised to make the whole setup process of RSAT easier. At the current state, it's quite cumbersome to get things running.

    Once all is installed, it works as intended. There are some basic rules which need to be covered
    • Create a dedicated LCS BPM library for RSAT only
    • Create a corresponding Azure DevOps project for your RSAT test plan
    • Don't record E2E scenarios - rather record small portions of a process (e.g create a customer, then create a sales order, then...)
    • Start with a small batch of recordings to create an E2E process
    Try to think small and once this works, grow your test library! If you want too much already in the beginning you're likely to end up frustrated and chasing errors.

    We test UI, ISV add ons and integrations as far as (technically) possible.

    Hope I could help

    Best
    Andreas

    ------------------------------
    Andreas Mörker
    Global Master Data Manager
    SIGVARIS
    ------------------------------



  • 3.  RE: Automated regression testing for D365

    TOP CONTRIBUTOR
    Posted Apr 26, 2019 02:10 PM
    Edited by Zvika Rimalt Apr 26, 2019 02:13 PM
    Thanks Andreas,

    Do I understand correctly from your message your company is in the process of implementing and has not gone live in production with the testing tools?

    Also  - in addition to run tests that verify transaction entry by users, can this tool also test interfaces?


    ------------------------------
    Zvika Rimalt
    Functional Consultant
    Vancouver BC
    ------------------------------



  • 4.  RE: Automated regression testing for D365

    Posted Apr 29, 2019 03:42 AM
    Yes, we are currently preparing the tool. Recording the processes is a heck of a business and takes much more time than MS tells you.
    We won't be testing in Prod. We have an identical UAT System that matches almost Prod (in terms of sizing). where we do run the RSAT tool.

    It also works on a dev box but is not recommended due to transaction data loads you want to test!

    You can test interfaces as well but you need to carefully think this through, where to start and how to trigger, etc.! Best would be do specifically draw the entry points and how to trigger.

    Also, the support from your interface provider in terms of testing environment needs to be in place.

    ------------------------------
    Andreas Mörker
    Global Master Data Manager
    SIGVARIS
    ------------------------------



  • 5.  RE: Automated regression testing for D365

    TOP CONTRIBUTOR
    Posted Apr 26, 2019 12:15 PM
    Hi Zvika,

    We've lined up a speaker about this for our upcoming AXUG meeting on June 6th.

    Details to come...

    Thanks,

    Chris

    ------------------------------
    Christopher Ho Yee
    Altius Consulting
    Richmond BC
    ------------------------------



  • 6.  RE: Automated regression testing for D365

    GOLD CONTRIBUTOR
    Posted Apr 27, 2019 07:18 AM
    + some very good insights here https://henrikmarxlarsen.wordpress.com

    ------------------------------
    Regards
    Rahul Mohta
    Advisor - #D365FO
    Real Dynamics
    California, USA
    ------------------------------



  • 7.  RE: Automated regression testing for D365

    Posted Apr 30, 2019 02:43 AM
    Zvika,

    We have spent significant resources in testing RSAT is presented and the results are not positive.  We have determined we can complete the regression testing as a team faster and more efficiently.  As a a warehouse and inventory control analyst, I can say I am at a meeting now to see how RSAT can integrate with WMS which could be a significant help to our business.

    As far as 3rd party tools, we are in the process of reviewing and testing a few, two of them are Axeptance https://axeptance.com/ and Rapise https://www.inflectra.com/Rapise/.

    ------------------------------
    William McLendon
    CRC Industries, Inc.
    Horsham PA
    ------------------------------



  • 8.  RE: Automated regression testing for D365

    TOP CONTRIBUTOR
    Posted Apr 30, 2019 08:55 AM
    Thanks William, please keep updating on this thread what you find out about WMS testing - we have the same requirement.

    I find it quite ironic that Microsoft on one hand pushes all customers to the "one version" path where you are forced to do much more upgrades every year (so tons more regression testing that you would have chosen to do), but yet not providing a very good solution for automated regression.


    ------------------------------
    Zvika Rimalt
    Functional Consultant
    Vancouver BC
    ------------------------------



  • 9.  RE: Automated regression testing for D365

    Posted May 06, 2019 09:43 AM
    ​I as well am looking to understand how the Microsoft proposed OneVersion Automated Testing Solution will be rolled out.  It appears that the solution is not ready for prime time and still weeks out before we can get our hand on it.  Are there any webinars, classes, sessions, etc... to attend to get up to speed with the configuration and start-up?

    ------------------------------
    Dennis Thimling
    Jasper Engines
    Jasper IN
    ------------------------------



  • 10.  RE: Automated regression testing for D365

    GOLD CONTRIBUTOR
    Posted May 31, 2019 09:22 AM
    I am also worried  about the extra Infrastructure needs, ongoing resource skill set we need to source

    ------------------------------
    Crystal Ahrens
    Solution Architect
    Kodak Alaris Inc.
    Albion NY
    ------------------------------



  • 11.  RE: Automated regression testing for D365

    Posted May 31, 2019 03:36 AM
    Edited by Artur Zyzak May 31, 2019 04:07 AM
    Hello Zvika,

    RSAT is doing better with its every new release but we already see it still requires a lot of patience, resources and dedicated management. It is also not really handling specific testing needs (like WMS testing mentioned above, or other Dynamics365 family products) for scripts reausability, data validations etc.

    You may look at  Executive Automats - Microsoft Dynamics AX / D365 Automated Testing Tool & Security Set-up
    ExecutiveAutomats.com remove preview
    Executive Automats - Microsoft Dynamics AX / D365 Automated Testing Tool & Security Set-up
    Unique Test Automation Tool for D365 and Microsoft AX. Security Set-up for Dynamics AX/365.
    View this on ExecutiveAutomats.com >

    this is a solution that brings all the features already. To be transparent I'm personally affiliated with this project, but as long as we already have super positive feedback from our Dynamics 365 clients, I think this is worth mentioning :)

    ------------------------------
    Artur Zyzak
    Head of Product Development
    XPLUS
    Warszawa
    ------------------------------



  • 12.  RE: Automated regression testing for D365

    GOLD CONTRIBUTOR
    Posted May 31, 2019 09:04 AM
    Great question , I just attended a tech talk on the subject by Microsoft and open some many more questions. with D365 seems like we need really documentation on this subject RSAT

    ------------------------------
    Crystal Ahrens
    Solution Architect
    Kodak Alaris Inc.
    Albion NY
    ------------------------------



If you've found this thread useful, dive deeper into User Group community content by role