Author Topic: Handling more than 1000 #Define entries  (Read 2560 times)

njlaw

  • Newbie
  • *
  • Posts: 10
    • View Profile
Handling more than 1000 #Define entries
« on: March 05, 2021, 02:15:47 PM »
Hello,

I have a program that continues to increase in size and complexity and am now out of #Define entries.  Since this is compiler-side and doesn't actually impact memory usage on the PLCs, 1000 seems like a rather arbitrary limit.  Is there any chance this could be increased?

And since I am sure that someone will be curious why we are using so many #Define entries, I'll try to explain.  We have a little computer (IoT gateway) that is deployed along side the PLCs.  It translates the content of the PLC memory to JSON and sends certain portions of it to the cloud live and certain portions on demand.  It is also able to take JSON documents from the cloud and use them to write to the PLC memory.  As such the program needs to be able to map between PLC memory and the JSON documents.  By using #Define entries for every variable, we are able to update memory addresses and mapping relatively easily: the PLC program stays the same and just by regenerating the #Define table from a database, we can move memory and preserve the mapping to JSON.

E.g. say we have,
DM32Arr_LUT1 | DM32[225]
DM32Arr_LUT2 | DM32[250]

Due to product / program revisions, we need to increase the size of the LUT1 lookup table.  With our system, I can just publish a new #Define table:

DM32Arr_LUT1 | DM32[225]
DM32Arr_LUT2 | DM32[270]

And I don't have to change the code in the TRiLOGI program at all.

Any thoughts or alternatives are welcome.

Best regards,

Nathanael

garysdickinson

  • Expert Member
  • Posts: 475
  • Old PLC Coder
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #1 on: March 06, 2021, 10:46:00 AM »
Nathanal,

I am not an expert on JSON, but I do have a method for making DMxx[] look like an array of data structures and not just a simple array of numeric values.

The approach that I have taken does use the #Define mechanism but does not need as many #Define entries as you are using.  I only use a #Define for each structure member, and the total size of the data structure.

If you are interested, I can post bits of my PLC code and #Define tables at some explanation for how I use the approach.  I have been using this approach for 10+ years, but have not posted it on the forum.

Best regards,

Gary Dickinson

njlaw

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #2 on: March 06, 2021, 11:50:15 AM »
Hi Gary,

If you would be willing to share some snippets, that would be appreciated.  I am already trying to use arrays and offsets (basically like pointer math in C) to reduce the need for #Defines where possible, but any other techniques would be helpful.

E.g.,
Code: [Select]
ingredientMatched = 0
FOR I = 0 TO 5
IF FP[fpSrcStart + CONST_MD_Ingredient_FPOffset + I * CONST_MD_Ingredient_FPLength + CONST_MD_IngredientDosage_FPOffset] > 0.0 OR FP[fpSrcStart + CONST_MD_Ingredient_FPOffset + I * CONST_MD_Ingredient_FPLength + CONST_MD_IngredientMinDosage_FPOffset] > 0.0 OR FP[fpSrcStart + CONST_MD_Ingredient_FPOffset + I * CONST_MD_Ingredient_FPLength + CONST_MD_IngredientMaxDosage_FPOffset] > 0.0 THEN
matched = 0
FOR J = 0 TO 5
IF TestBit(DM16_ActiveComponents1, CONST_ActiveComponents1_Ingredient1 + J) AND CompDMMem(dmSrcStart + CONST_MD_Ingredient_DM16Offset + I * CONST_MD_Ingredient_DM16Length + CONST_MD_IngredientName_DM16Offset, CONST_Ingredient1Name_DM16Addr + J * 8, 8) = 0 THEN
matched = 1
SetBit ingredientMatched, J
ret = CopyDM(dmSrcStart + CONST_MD_Ingredient_DM16Offset + I * CONST_MD_Ingredient_DM16Length, dmDstStart + CONST_MD_Ingredient_DM16Offset + J * CONST_MD_Ingredient_DM16Length, CONST_MD_Ingredient_DM16Length)
ret = CopyFP(fpSrcStart + CONST_MD_Ingredient_FPOffset + I * CONST_MD_Ingredient_FPLength, fpDstStart + CONST_MD_Ingredient_FPOffset + J * CONST_MD_Ingredient_FPLength, CONST_MD_Ingredient_FPLength)
ENDIF
NEXT
IF matched = 0 THEN
SetBit DM16_StageMDErrFlags2, (CONST_StageMDErrFlags2_Ingredient1DoesNotMatch + I)
ENDIF
ENDIF
NEXT

In the above code, fpSrcStart points to the FP address at the start of the recipe that has been pushed from the cloud and fpDstStart points to the FP address of the recipe used in live batches.  (dmSrcStart and dmDstStart work the same way.)  It then goes through all the ingredients in the recipe and matches them to the ingredients loaded into the machine slots, and copies the dosage and other parameters from the recipe to the correct slot for operation.

This way, I don't use #Defines for all the components of each recipe (5 can be loaded into the PLC from the cloud at once), but only need the starting location and relative offsets for all six data structures (5 recipes from the cloud that can be selected at run time by the operator and the live recipe).

Best regards,

Nathanael
« Last Edit: March 06, 2021, 11:52:27 AM by njlaw »

garysdickinson

  • Expert Member
  • Posts: 475
  • Old PLC Coder
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #3 on: March 06, 2021, 03:42:22 PM »
Nathanael,

You have figured out about the same solution to handling arrays of structures in TBASIC as I did.  So I may not be able to reduce the number of define table entries that you desire.

My approach is slightly different. This is an exert from the #Define table for a single structure. My application has an array of 8 of these structures


Code: [Select]
#   #Define Name        Value
45  FillerArrayBase     1
46  FillerArraySize     10
47  FillerStatus        DM32[y+0]
48  FillerCapacity      DM32[y+1]
49  FillerVolume        DM32[y+2]
50  FillerUllage        DM32[y+3]
51  FillerDvDt          DM32[y+4]
52  FillerTtlDvDt       DM32[y+5]
53  FillerBOL           DM32[y+6]
54  FillerDeliveryVol   DM32[y+7]
55  Delivered           DM32[y+8]
56  FillerReserved09    DM32[y+9]

FillerArrayBase is the index into DM32[] for the first entry of the first data structure.
FillerArraySize is the size of the data structure in 32-bit words.
There are 8 of these structures in the array. The maximum number of structures is defined as MaxFillerN. This allows me to change the size
of the array without having to change any of the CFs that act on the array of structures (there are no hard-coded values in my code for this sort of thing).
 
What I do differently that you is the computation of the index into DM32[] to access a individual member of a structure. I compute the index to the the first member
of the structure that I am interested in and assign this to the variable "y".  This keeps me from making simple mistakes in my coding that will be hard to find during debug.

The variable "y" is set to the starting address of of the "current" data structure once on each loop through the for..next structure.

Code: [Select]
' UpdateStatus - Now that all of the tank measurements have been made, 
'   this function will update the filler status information for each filler.
'
for FillerIndex = 0 to MaxFillerN       ' for each fill point
    ' The variable, "y", is the index into the array of data structures that
    '   represent the state of an individual filler.
    '
    y = FillerArrayBase + FillerIndex * FillerArraySize
   
    ' a mountain of code was deleted out of this function. The missing code
    '   assigned values to the XXXXTemp variables based on the state of 16
    '   fuel tanks that may be connected to the system. the XXXXTemp variables
    '   are defined as locations in DM32[] with the #Define mechanism.
    '
    FillerStatus    = StatusTemp
    FillerVolume    = VolumeTemp
    FillerUllage    = UllageTemp
    FillerDvDt      = DvDtTemp
    FillerTtlDvDt   = TotalizerDvDtTemp
    FillerCapacity  = CapacityTemp

next    ' FillerIndex (next filler)

OK. What might I suggest that you do to cut down on the number of #Define entries?  It sort of depends on how you have structured your program.

If you have 5 different recipes, do you save all five in the EEP?  As you appear to be using a Fx or Tile-based PLC you may have as much as 11K 16-bit words of EEP that you can store the recipes. If this is true, then you only need to access the current active recipe. You could copy the active recipe into a fixed location in DM32[] or you could access the EEP for the active recipe directly.  Load_EEP32[(y+n)] ... where y is that stating index into EEP32 for the active structure and n is the offset within the array.

Another thing that you could do to simplify your life is to store all of the data in the same place. I notice that you have some 32-bit float values and some 32-bit integer values. If you have only a few float values you could:

1. Convert them to scaled integers and save them in DM32[]. Lets say that the float value is 32.467 and your system only needs the number to be accurate to +/-0.1 then save the float      DM32[n] = round(32.467,1) * 10    This is a scaled integer. When you need to use it covert it to what ever units your system actually uses.
2. Save the float value DM32 as a bit pattern.  DM32[n] = Float2Bits(32.467).  When you need to use the value as a float convert it back   BitsFloat(DM32[n]).

By not having to split your recipes between DM32[] and FP[] might make your life a bit simpler(???).

Oh, and you can delete all uses of "THEN" from your if..else statements. "THEN" is purely optional and is purely useless.  Won't save you any execution time or reduce the number of #defines but it might make your .PC7 files a few bytes smaller.

Best regards,

Gary Dickinson
« Last Edit: March 06, 2021, 08:12:07 PM by garysdickinson »

njlaw

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #4 on: March 08, 2021, 09:42:07 AM »
Hi Gary,

Thank you for all the tips!  In particular, I hadn't considered using another variable reference inside the Value of a #Define entry.  I've also been avoiding using the EEPROM, but might need to look at that since I'm down to 240 free DM addresses and 90 free FP addresses.

Thanks again!

Nathanael Law

garysdickinson

  • Expert Member
  • Posts: 475
  • Old PLC Coder
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #5 on: March 08, 2021, 11:30:24 AM »
Hi Nathanael,

I am glad I got you thinking.

Another resource is the file system that is supported by all of the modern PLCs (for the FMD and Nano-10 you have to add the FRAM-RTC-256 module). The file system allows up to 256K bytes of text data.  The 256K can be organized in a series of files. 

These files can be uploaded, downloaded and deleted by use of FTP.

The PLC can create, read, write, append, delete these files. The PLC can check for the existence of a file and determine file size.

I use the file system with my clients to allow remote configuration of their systems.  Additionally I have the PLCs generate a set of files that describe the current system configuration on each power up/reboot event.

The current files can be used as documentation.  One can edit the files with a text editor and upload the edited file back to the PLC to modify its configuration.

To store 4000 32-bit data words in hexadecimal would require about 36K bytes.  DM32 holds 2000 32-bit integers and FP holds 2000 floats that adds up to 4000 32-bit values. You could store 7 copies of DM32[] and FP[] in 256k of text space! 

TBASIC is actually pretty good at working with strings.

More things to think about.

Gary d

support

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3135
    • View Profile
    • Internet Programmable PLCs
Re: Handling more than 1000 #Define entries
« Reply #6 on: March 10, 2021, 11:06:26 PM »
Hi Nathanael and Gary

Thank you for the robust discussion about getting around the 1000 table entries limit on i-TRiLOGI. You are right that the 1000 number is what we think should be more than enough for most TBASIC program and we didn't want the table to become so large that it is hard to locate a defined item.

We can consider expanding the number of allowable table entries in the upcoming release of new i-TRiLOGI version 6.7 and 7.4 with a number of user interface improvement.   How many more entries would you need in the #Define table?

What PLC model are you using?
Email: support@triplc.com
Tel: 1-877-TRI-PLCS

garysdickinson

  • Expert Member
  • Posts: 475
  • Old PLC Coder
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #7 on: March 26, 2021, 04:14:42 PM »
I use lots of #define entries.  My #1 usage is to create named variables from DM[], DM32[] and FP[].

I export the #defines to Excel so that I can build the register address to allow the Wientek HMI to access PLC variables and RELAYS.  I make less mistakes in getting the devices to talk to each other by this approach.

I miss the concept of data structures that modern programming languages support, but TBASIC does not.  I use the #define mechanism to allow me to build data structures and arrays of structures.

So I don’t know the right answer to how many #defines.

But I have a suggestion/request on how to reduce the number of #defines that I use.

Provide access to EEPROM data using array notation.

something like this:

EEP32[n] = 6  :  A = EEP32[n]

That would allow me to define a variable stored in EEP once.   #define Diameter  EEP32[23]

Now I can write code like this:

Diameter = 12

if Diameter > 2
    ' do something useful
    '
end if

Currently, if I limit my self to only using a single define to access EEP I can do something like this:

#define Diameter  23

The usage looks like this:

SAVE_EEP32 12, Diameter
If LOAD_EEP32(Diameter) > 2
    '
end if

To assign a value to EEP I have to use a statement.  To access a value in EEP I have to use a parameterized function call.

The single define approach requires, that I have to be very careful in writing my code.  EEP access for integers can be 8,16 and 32 bit unsigned integers. EEP access can support floating point and string data, also. This creates a lot of ways to write buggy code.  TBASIC cannot detect that you are accessing an EEP variable in one spot as an 8 bit unsigned int and in another place as a 32-bit float.  But you can easily write code like this...

If EEPROM could be accessed as EEP8[], EEP16[], EEP32[], EEP#[] and EEP$[] then I can use a single #define for all access to a single EEP resident variable.  It is harder for me to make simple mistakes that TBASIC can’t detect. Saves me from trying to convince my clients that my code is not buggy but rather feature laden.

Gary d


support

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3135
    • View Profile
    • Internet Programmable PLCs
Re: Handling more than 1000 #Define entries
« Reply #8 on: March 30, 2021, 08:22:50 AM »
Hi Gary,

Thank you for the feedback and suggestions. We will consider what you have suggested while avoid confusing user of the difference between the EEPROM memory and the DM memory. Although in our PLCs that use FRAM such as any SmartTILE-Fx based PLC, the reading and writing to  EEP are almost as fast as DM. In some other models that use real EEPROM the performance of reading and writing to EEPROM are very different from that to the DM so we want user to be well aware of the differences and not use them interchangeably. We will see what is the best way to achieve what you have suggested.
« Last Edit: March 30, 2021, 07:57:23 PM by support »
Email: support@triplc.com
Tel: 1-877-TRI-PLCS

njlaw

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #9 on: March 30, 2021, 08:36:28 AM »
How many more entries would you need in the #Define table?

I think it would be difficult to put a definite number to that: I've lost track of the number of times I thought, "I'll never use that much RAM, CPU, etc."  A total of 2000 (1000 more) would certainly get us past the immediate squeeze for quite some time.

What PLC model are you using?

The program that we are running into the #Define limit with is running on a Fx1616-BA.  We have other programs on Nano-10s, but the complexity of those programs is significantly less, and we're around 250 #Define entries there.

support

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3135
    • View Profile
    • Internet Programmable PLCs
Re: Handling more than 1000 #Define entries
« Reply #10 on: March 30, 2021, 08:16:23 PM »
We should have no problem expanding the #Define table to 2000 entries. If you require the version urgently please contact support@triplc.com and we could arrange to send you a pre-release version of new i-TRiLOGI 7 that can support 2000 entries in the #Define table along with many other new features.
Email: support@triplc.com
Tel: 1-877-TRI-PLCS

njlaw

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #11 on: June 28, 2021, 06:10:00 PM »
I'm just following up to say that I appreciate the increase of the #Define table limit in v7.4!

Thank you!

Nathanael Law
« Last Edit: June 29, 2021, 09:23:34 AM by njlaw »

support

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3135
    • View Profile
    • Internet Programmable PLCs
Re: Handling more than 1000 #Define entries
« Reply #12 on: June 30, 2021, 10:11:06 AM »
i-TRiLOGI 7.4 is released to support all the new features (the integrated 16-key keypad and a 128 x 64 pixel OLED display as well as WiFi configuration) in the new Wx100 PLC (https://triplc.com/wx100.htm), but is also tested to work with all other Super PLCs such as the Nano-10, FMD, Fx and any SmartTILE-Fx based custom PLCs.

In response to your feedback we decided to expand the #DEFINE table to 2000 entries when the final version is released.

One more feature that you may find helpful to manage large #DEFINE table is to use the Excel file "ExcelDefineTable.xlsm" in the "C:\TRiLOGI\TL74" folder.

You can export the #Define table to a text file and the "Import"  macro in the spreadsheet can import the text file into Excel, allowing you more powerful tools in Excel to manage the table. After the table has been updated, click the "Export" button to run a macro to export the updated file back to a text file which can then be imported into i-TRiLOGI 7.4. 

We would like to thank the Expert Member of this forum - Gary Dickinson for his suggestion and help in creating the "ExcelDefineTable.xlsm"
« Last Edit: June 30, 2021, 10:27:25 AM by support »
Email: support@triplc.com
Tel: 1-877-TRI-PLCS

Roman Jay Almaza

  • Newbie
  • *
  • Posts: 5
    • View Profile
Re: Handling more than 1000 #Define entries
« Reply #13 on: August 24, 2021, 11:08:48 PM »
Hi Gary,

This is a cool way to improve handling more than 1000 #Define entries. It really got me thinking about how I can improve my system, I am still a noob to this thread but I have been wondering the same thing just like with Njlaw.
Again thank you.