Modern Natural Development with NaturalONE – Legacy Coder Podcast #3

Modern Natural Development with NaturalONE

Why would anyone dismiss the good old Natural editor and switch over to NaturalONE? Let’s find out what features NaturalONE has to offer in the third episode of the Legacy Coder Podcast.

Why should you switch over to NaturalONE?

  • Increased developer productivity
    • Use Copy and Paste (duh!) for high quality Natural programming 😉
    • Syntax Highlighting will help you understand the source code faster.
    • Make use of intelligent search and replace features with regular expressions.
    • Code Completion makes you type way less than before.
      Code Completion in NaturalONE
    • Code Navigation (jump to a variable definition etc.) lets you explore the code base with a few keystrokes.
    • You can use Code Folding for focussing on the really important stuff.
    • Multiple compile errors are displayed instantly without uploading the source to the server.
      Multiple compile errors in NaturalONE
  • Repository based development
    • Finally develop software like the cool kids do!
    • Every code change is documented and can be rolled back with just a few commands. It can even be attached to an issue tracker like Jira or Redmine.
      Git integration in NaturalONE
    • You can try out new things in an isolated environment.
    • You can automate your coding with code generators like Xtext.
  • Modernize your development environment
    • Share knowledge and open up towards other (maybe younger) developers (Java etc.).
    • Be and stay attractive as a company to new employees.
    • Automate your build process
    • NaturalONE is Software AG’s platform of choice for the future.
    • The old Natural editor will be shut down!
  • Extendable with additional plugins
    • Software AG’s innovations will only take place in NaturalONE (e.g. Code Coverage, Screen Testing).
    • Attach your IDE to an issue tracker like Jira or Redmine with Mylyn.
    • Use Snipmatch for even less typing. Plugin Snipmatch used in NaturalONE
    • Create your own extensions (e.g. NatUnit) for whatever requirement you have.
      Our own NatUnit plugin used in NaturalONE

Recommended reading

  • NaturalOne Debugging

Links

Automating Your Natural Build Process – Legacy Coder Podcast #2

Automating Your Natural Build Process

How can you automate the build process for your application based on Software AG’s Adabas and Natural? I talk about our journey towards a completely automated build from scratch after each push to Git in the second episode of the Legacy Coder Podcast.

How to automate the build process for an Adabas/Natural application

  • Why would I want to automate the build process?
    • What hurts should be done more often!
    • No more long integration phases before each release!
    • You’ll get a faster time to market for new features.
    • You can get rid of knowledge silos concerning the release of your application.
  • What steps may need to be automated?
    • Upload (transfering the Natural source code to the server)
    • Compile (CATALOG, STOW, CATALL etc.)
    • Unit Tests (e.g. with NatUnit, see Unit Testing Natural Applications)
    • Deployment (the actual release of the new version on the target stage)
    • Restarting RPC servers or whatever individual steps might be necessary in your environment
  • What tools are available for automating the Natural build process?
    • It all starts with Git or any other versioning control system (Subversion, TFS, CVS).
    • You can use Jenkins for Continuous Integration.
    • Software AG’s command line tools (ftouch etc.) can be scripted.
    • Even the CATALL can be scripted by calling into Natural BATCHMODE.
    • NaturalONE’s ant scripts for uploading and compiling Natural sources.
    • Any additional tool that you might need (Bash scripts etc.)
  • How can I get started?
    • Make your project fully compilable. There can be no compile errors whatsoever.
    • Start with a single step, e.g. automate only the compile step first.
    • Script everything (e.g. with Bash) right from the beginning. You can easily call these scripts from Jenkins later on.
    • Hook Jenkins up to your source code repository. It provides plugins for every major system (e.g. Git, Subversion).
    • Try to automate the build with Jenkins. It’s completely free and easy to setup. Perhaps ask a Java colleague if he can help you.
  • How can you go even further than what’s possible today?
    • Try to automate imports of Adabas data that’s important for your application.
    • Try to automate Adabas schema migrations.
    • Implement feature toggles for more fine grained control over your application’s functionality at runtime.

Recommended reading

Links

Unit Testing Natural Applications – Legacy Coder Podcast #1

Unit Tests for Natural Applications with NatUnit

Unit testing with Software AG’s Natural – is that even possible? Let’s find out in the very first episode of the Legacy Coder Podcast.

Unit testing Natural applications

  • Why would I want to unit test my Natural code?
    • You get instant feedback whether your code works. That’s fun! 😀
    • You get a safety net for refactoring and optimizing your Natural code.
    • You can catch regression as early as possible. That makes evolving your code base easy and painless.
    • Unit tests can be integrated into your build process (Continuous Integration, Continuous Delivery, Continuous Deployment).
    • Unit tests are the first step on your way to a fully automated build process.
  • What tools are available for unit tests with Natural?
    • Subprogram testing within NaturalONE
      • Only subprograms can be tested.
      • Different “language” for defining your tests.
      • Tests have to be defined manually from a GUI.
      • Tests cannot be executed from within Natural itself.
      • Proprietary software maintained by Software AG.
      • Professional support by Software AG is available.
    • NatUnit – an open source framework for unit testing with Natural
      • Everything that can be called from a subprogram can be tested (subprogram, subroutine, function).
      • Tests are written in the same language that is used for production code.
      • Tests can be written as Natural code with full IDE support.
      • Tests can be executed within any Natural runtime.
      • Open source project that can be extended by everybody – including yourself.
      • There’s no professional support for the framework.
  • What do I have to do differently when writing testable Natural modules?
    • You need more fine grained modularization!
    • You have to decouple the User Interface, Database access, and the core Business Logic from each other.
    • You need to extract the functionality to test into its own callable module.

NatUnit in action

Here are a few screenshot of our NatUnit Eclipse plugin in action.

You can run a NatUnit test from within the IDE.

NatUnit Eclipse Plugin: RunAs NatUnit Test

If the tests are successful, you instantly see the famous “green bar”.

NatUnit Eclipse Plugin: Successful Test

And if the tests are broken, you’ll definitely notice that, too. 😉

NatUnit Eclipse Plugin: Failing Test

Recommended reading

Links

Introduction to the show and the host – Legacy Coder Podcast #0

Legacy Coder Podcast Episode 0: Introduction to the show and the host

Who is this guy and what the heck does he know about Software AG’s Adabas and Natural? I’ll show you in the pre-episode zero of the Legacy Coder Podcast.

Introduction to the podcast and myself

  • My experience with Adabas and Natural.
    • I’m programming with Natural since 2003.
    • I’m the president of the International Natural User Group since 2016.
    • I co-authored NatUnit (as part of my Master’s thesis) and L4N.
    • My main job is integrating Adabas/Natural into a modern SOA with webMethods Integration Server.
    • In the last few months I automated and modernized the Natural build process with Git, Jenkins, Java, and NaturalONE.
    • My biggest Natural project so far was the migration of an insurance system from Oracle/Java to Adabas/Natural.
    • I frequently speak at local and international user groups and conferences (e.g. Innovation World, Innovation Tour, IUG).
    • Our Natural code base: ~6 million LOC, ~19.000 modules.
  • My professional experience.
    • I studied business computer science (Bachelor’s degree) and software engineering (Master’s degree).
    • I train new hires and apprentices at ALTE OLDENBURGER Krankenversicherung AG in Germany.
    • I also use a few additional languages: Java, C#, PHP, Ruby.
    • I’m a software architect since 2014.
    • I work as an associate professor for programming and software engineering at PHWT since 2009.
  • My personal life.
    • I’m married.
    • I have two dogs.
    • I like taking photos.
    • I run a podcast for aspiring software developers in German.
    • I organize my own user group and conference for software developers.
    • You can find me almost everywhere: LinkedIn, Twitter, Instagram etc.

Links

Exception-like error handling in Software AG’s Natural

Error handling in Software AG’s Natural can be done in a way that resembles Exception handling in object-oriented languages like Java.

throw

Instead of throwing an Exception, you raise an error simply by assigning a value to the system variable *ERROR-NR. As soon as a statement like the following is executed, the current program flow is interrupted and the nearest ON ERROR block is executed.

*ERROR-NR := 1234

In fact, we use exactly this feature for raising assertion errors in NatUnit.

catch

You can handle a Natural error in an ON ERROR block anywhere inside your code. Just like an Exception travels up through the call stack to get caught in the nearest try-catch block, a Natural error is handled in the nearest ON ERROR block.

Here’s an example of an ON ERROR block that exits the current module and marks the error as handled:

ON ERROR
    /* do something about it */
    ESCAPE MODULE
END-ERROR

catch (SpecificException e)

You can only define a single ON ERROR block in each Natural module. So if you need to handle specific errors in a different way, you need to have some kind of distinction logic like this:

ON ERROR
    IF *ERROR-NR EQ 1234
        /* do something about it */
        ESCAPE MODULE
    END-IF
END-ERROR

Or if you need to distinguish between multiple errors:

ON ERROR
    DECIDE ON FIRST VALUE OF *ERROR-NR
        VALUE 1234
            /* do something about it */
            ESCAPE MODULE
        VALUE 1235
            /* do something about it */
            ESCAPE MODULE
        NONE IGNORE
    END-DECIDE
END-ERROR

re-throw

If you can’t handle the error in an ON ERROR block, but you want to log it or do something else with it before letting the next ON ERROR block handle it, you don’t need to do anything at all, because that’s the default behaviour.

However, if you exit the ON ERROR block with any statement from the following list, the error is marked as handled and the normal control flow (in the calling module of the module containing the ON ERROR block) is continued. So be sure not to exit the block with any of these statements.

Exiting from an ON ERROR Block:
An ON ERROR block may be exited by using a FETCH, STOP, TERMINATE, RETRY, ESCAPE ROUTINE or ESCAPE MODULE statement. If the block is not exited using one of these statements, standard error message processing is performed and program execution is terminated.

Here’s an example of such a “re-throw”:

ON ERROR
    IF *ERROR-NR EQ 1234
        /* log the error */
        /* DON'T exit with `FETCH`, `STOP`, `TERMINATE`, `RETRY`, `ESCAPE ROUTINE` or `ESCAPE MODULE` */
    END-IF
END-ERROR

Checking which error occurred

Even if you “handle” the Natural error in an ON ERROR block, e.g. by using ESCAPE MODULE, the system variable *ERROR-NR isn’t reset to 0. You need to do that yourself, if you need to. If you don’t, the variable can be used in the calling module to check whether an error (that was handled) occured. By the way, the system variable *ERROR-LINE contains the line number of the statement that raised the error.

CALLER

CALLNAT 'CALLEE'
IF *ERROR-NR NE 0
    WRITE 'Error' *ERROR-NR 'occurred in line' *ERROR-LINE 'while calling CALLEE'
    /* prints: "Error     1234 occurred while calling CALLEE" */
END-IF
END

CALLEE

*ERROR-NR := 1234
ON ERROR
    ESCAPE MODULE
END-ERROR
END

If you don’t want any caller to know that an error occurred, simply reset *ERROR-NR:

ON ERROR
    RESET *ERROR-NR
    ESCAPE MODULE
END-ERROR

Global error handler (like a try-catch in main())

You can define a global error handler by setting the system variable *ERROR-TA to the name of a Natural module. In case of an error, Natural automatically calls this module (which has to be a program) and puts information about the error on the stack. The system variables *ERROR-NR and *ERROR-LINE will be reset at this point, so the error handler has to read the information from the stack with INPUT.

CALLER

*ERROR-TA := 'HANDLER'
CALLNAT 'CALLEE'
END

CALLEE

*ERROR-NR := 1234
END

HANDLER

DEFINE DATA LOCAL
1 #ERROR-NR           (N5)
1 #LINE               (N4)
1 #STATUS-CODE        (A1)
1 #PROGRAM            (A8)
1 #LEVEL              (N2)
1 #LEVELI4            (I4)
1 #POSITION-IN-LINE   (N3)
1 #LENGTH-OF-ITEM     (N3)
END-DEFINE

/* read error information from stack */
INPUT #ERROR-NR #LINE #STATUS-CODE #PROGRAM #LEVEL #LEVELI4

WRITE #ERROR-NR #LINE #STATUS-CODE #PROGRAM #LEVEL #LEVELI4 #STATUS-CODE
WRITE *ERROR-NR *ERROR-LINE

END

Output:

1234    10 O CALLEE     2           0 O
     0     0

For more information about the error information on the stack take a look at the section Using an Error Transaction Program in the Natural documentation.

Getting more information about errors programmatically

If you need to find out more about the current error, e.g. in your ON ERROR block, there are quite a few User Exits that deal with errors:

  • USR0040N: Get type of last error
  • USR1016N: Get error level for error in nested copycodes
  • USR2001N: Get information on last error
  • USR2006N: Get information from error message collector
  • USR2007N: Get or set data for RPC default server
  • USR2010N: Get error information on last database call
  • USR2026N: Get TECH information
  • USR2030N: Get dynamic error message parts from the last error
  • USR3320N: Find user short error message (including steplibs search)
  • USR4214N: Get program level information

How to find the physical file path of the current FUSER of a Natural runtime

Here’s a short subroutine for reading the physical file path of the current FUSER of a Natural (from Software AG) runtime. I’m not sure if it works on a mainframe, but it definitely runs on a Linux system.

The subroutine returns the following information, if it runs successfully:

P-FUSER-PATH /home/macke/fuser
P-RC 0

Otherwise the return code P-RC will have a value other than zero.

It uses two user exits:

  • USR6006N: Get path to system file
  • USR2013N: Get SYSPROF information

USR2013N reads the information about the current FUSER and returns its DB-ID and File Number. And USR6006L takes these two inputs and returns the physical file path of the FUSER.

Subroutine GET-CURRENT-FUSER-PATH

**************************************************************************
*
*  File: GET-CURRENT-FUSER-PATH (VNGFUPAT)
*
*  Reads the physical file path for the current FUSER.
*
*  Tags: FUSER, UserExit
*
*  Parameters:
*    -
*
*  Returns:
*    P-FUSER-PATH - File path for the current FUSER.
*    P-RC - Return code
*
**************************************************************************
DEFINE DATA
*
PARAMETER
*
01 P-RC (I4) BY VALUE RESULT
01 P-FUSER-PATH (A) DYNAMIC BY VALUE RESULT
*
LOCAL
*
* Get path to system file
01 USR6006L
  02 INPUTS
    03 SYSF-DBID (I4)
    03 SYSF-FNR (I4)
  02 OUTPUTS
    03 SYSF-PATH (A253)
    03 RESPONSE-CODE (I4)
    03 INFOTEXT (A65)
01 EXTENSIONS (A1/1:1)
*
* Get SYSPROF information
01 USR2013L
  02 OUTPUTS
    03 FILENAME (A12/1:50)
    03 DBID (P5/1:50)
    03 FNR (P5/1:50)
    03 DBNAME (A11/1:50)
    03 AMOUNT (P4)
*
01 #INDEX (I4)
*
01 #FUSER-DBID (N8)
01 #FUSER-FNR (N8)
*
END-DEFINE
*
DEFINE SUBROUTINE GET-CURRENT-FUSER-PATH
*
RESET P-FUSER-PATH P-RC USR6006L USR2013L EXTENSIONS(*) #FUSER-DBID #FUSER-FNR
*
CALLNAT 'USR2013N'  USR2013L
*
FOR #INDEX = 1 TO USR2013L.AMOUNT
  IF USR2013L.FILENAME(#INDEX) EQ 'FUSER'
    #FUSER-DBID := USR2013L.DBID(#INDEX)
    #FUSER-FNR  := USR2013L.FNR(#INDEX)
  END-IF
END-FOR
*
IF #FUSER-DBID EQ 0 OR #FUSER-FNR EQ 0
  P-RC := 1
  ESCAPE MODULE
END-IF
*
USR6006L.SYSF-DBID := #FUSER-DBID
USR6006L.SYSF-FNR  := #FUSER-FNR
*
CALLNAT 'USR6006N' USR6006L EXTENSIONS(*)
*
P-FUSER-PATH := USR6006L.SYSF-PATH
P-RC := USR6006L.RESPONSE-CODE
*
END-SUBROUTINE
*
END

How to export all mapped environments from Natural Studio (SPoD)

A colleague of mine wanted to export all the mapped environments from Natural Studio (SPoD). As we have quite a lot of different environments due to our complex staging concept, manually re-creating this list would be cumbersome.

Map an environment in Natural Studio

Long story short: we didn’t find a way to export the environments from within Natural Studio. However, after a quick search I found this file: %PROGRAMDATA%\Software AG\Natural\6.3\Prof\%USERNAME%.PRU (which translates to C:\ProgramData\Software AG\Natural\6.3\Prof\MACKE.PRU in my case). It contains all the mapped environments in a text format:

Mapping2 = MAP THEHOST 2720 ALIAS=My environment; MACKE * * fuser=(22,123) ;CONNECTED=FALSE

Here’s a quick regular expression to filter the needed parts from the string:

MAP (.+) ([0-9]+) ALIAS=([^;]+); ([^ ]+) \* \* ([^;]*);.*
  • Group 1: Host name (THEHOST)
  • Group 2: Server port (2720)
  • Group 3: Environment name (My environment)
  • Group 4: User ID (MACKE)
  • Group 5: Session parameter (fuser=(22,123))

SOA-fying a Monolith – Innovation World 2015

My talk for Software AG’s Innovation World 2015 in Las Vegas got accepted and is already visible on the agenda: Lessons Learned from SOA-fying a Monolithic Legacy Application.

Logo Innovation World 2015

How do you modernize a monolithic legacy application to meet the requirements of today’s service-oriented world? In this talk, Stefan Macke shares his insights from SOA-fying a 15-year-old Adabas & Natural insurance application with the help of webMethods Integration Server, a bunch of unit tests and a domain-specific language for creating a canonical data model. He presents technical, architectural as well as organizational lessons learned from the modernization project.

Avoiding Natural Error 3009 (Adabas Timeout) in an RPC server

I published several Natural subprograms as a service in webMethods Integration Server via the EntireX adapter which uses a Natural RPC server on our SuSE Linux server to access the Natural modules. After a Natural service has not been called from the outside for a certain period of time (about 1 hour), the next call to the service always resulted in a Natural Error 3009:

NAT3009: Last transaction backed out of database :1:. Subcode :2:.

The Software AG documentation describes how to fix this problem: Avoiding Error Message NAT3009 from Server Program. However, there are a few additional things you need to check, to make sure that the solution works. Here is what I needed to do to make it work:

  • Copy NATRPC39 from library SYSRPC to library SYSTEM in your FUSER (System Libraries, not User Libraries!).
  • Edit SYSTEM.NATRPC39 and make sure every database gets pinged correctly:

    DEFINE DATA LOCAL
    01  ACB
      02  ACB-TYPE                      (B1) INIT<H'30'>
      02  ACB-FILL                      (B1)
      02  ACB-COMMAND                   (A2) INIT<'RC'>
      02  ACB-CID                       (A4) INIT<H'FFFFFFFF'>
      02  ACB-FILEID                    (B2)
      02  ACB-RSP                       (B2)
      02  ACB-ISN                       (B4)
      02  ACB-ISNL                      (B4)
      02  ACB-ISNQ                      (B4)
      02  ACB-FBL                       (B2)
      02  ACB-RBL                       (B2)
      02  ACB-SBL                       (B2)
      02  ACB-VBL                       (B2)
      02  ACB-IBL                       (B2)
      02  ACB-COP1                      (A1)
      02  ACB-COP2                      (A1)
      02  ACB-ADD1                      (A8)
      02  ACB-ADD2                      (A4)
      02  ACB-ADD3                      (A8)
      02  ACB-ADD4                      (A8)
      02  ACB-ADD5                      (A8)
      02  ACB-CMDT                      (B4)
      02  ACB-USER                      (A4)
    01  REDEFINE  ACB
      02 ACB-80                         (A80)
    END-DEFINE
    *
    RESET INITIAL ACB
    ACB-RSP := 1 /* DBID to ping
    CALL 'CMADA' ACB-80
    *
    RESET INITIAL ACB
    ACB-RSP := 2
    CALL 'CMADA' ACB-80
    *
    * add additional databases here
    *
    END
    
  • Check the setting SRVWAIT in the NATPARM of the RPC server. It needs to be greater than 0 and less than the TNA* settings for the Adabas database, e.g. 60 (seconds). The TNA* settings can be found in the database’s configuration file adanuc.prm.

  • Check the setting SERVER-NONACT in the Attribute File of the EntireX broker. It also needs to be less than the TNA* settings, e.g. 10M.

To make sure that NATRPC39 really gets called you may add a WRITE WORK part to its source like this:

DEFINE WORK FILE 1 '/tmp/NATRPC39.log' 
    TYPE 'UNFORMATTED' ATTRIBUTES 'APPEND'
#TIME := *TIMX
WRITE WORK 1 VARIABLE #TIME H'0A'
CLOSE WORK 1

In NATRPC39 be sure to only call modules in SYSTEM (as the Steplib chain is not evaluated) and check that it does not throw any runtime errors whatsoever! Any Natural error in NATRPC39 is silently ignored and not logged anywhere (or at least I found no log). To enable tracing for the RPC server to check whether the SRVWAIT works as expected (see Using the Server Trace Facility), you need to modify its NATPARM file: set TRACE to 2 and Trace on error to OFF. Then set the Report Assignment for Report 10 to Device LPT10 and set the Physical Output Device for Device LPT10 to something like this: /bin/sh -c cat>>/tmp/rpctrace.log. Be aware that the RPC server caches the log information. So you may need to deregister the RPC server after a few minutes to have it flush its log to the file.