I think I've run into a memory limitation or bug when using the MathcadAllocate() function provided by Mathsoft for use in user-defined routines.
I have written several user-defined functions, compiling them into DLLs with Microsoft Visual C++ 6.0. They have all seemed to work just fine.
Now, however, I'm trying to implement a rather complicated operation in my "function", which takes input parameters from my Mathcad sheet and performs a two-point-boundary-condition solution for a system of many differential equations.
When I run the code, it appears to overwrite memory spaces assigned early in the process - suggestive of a memory leak somewhere. When I use the same routines, compiled in a separate executable, they work normally. It is only when compiled for use with Mathcad that they misbehave. The only substantive change in the code when moving to Mathcad, other than adding the DLL entry point "wrapper", is that MathcadAllocate() takes the place of malloc() in the memory allocation routines.
Since the problem appears to be a memory leak that shows up only when moving to the DLL version, this suggests to me that the problem lies in MathcadAllocate().
I recognize that bugs of this kind can be notoriously difficult to diagnose, and that they often result from poor code. I cannot _guarantee_ that my code is clean, but I have pored over it and tested it in several ways. It's time now to ask you all (Mathsoft engineers - are you reading this??) whether you've come across anything like it.
Your explanation is jargon to me, you are too advanced. Don't think I would like entering the subject, however 3 sparks come back: 1_Some time ago, Hubert gave us the maximum array allocation: what was it 2 millons ? 8 millions ? 2_Somewhere, about numerical derivatives, Mathcad says "Table will display only 50 first values". So, for more values Mathcad is using some magic twist of its own. 3_In the Romberg numerical integration, it tabulates only 32 values but it can split up to 94. If you tabulate for the 94 values what you see in the 32 places table are the remaining of the values. Here again, mathcad is doing right but not apparent to the user. _________________ Those 3D with lot of points, you will never see the entire array of values, but the plot is OK. Eventually, the pointer you are talking about may look misplaced but calculations are right. Sorry for the perturbation.
I've also encountered memory allocation errors since the introduction of Mathcad2001. I've observed that when I allocate a complex return array using MathcadArrayAllocate I get different types of behavior depending on the array size. If I use the VC++ debugger and look at the data pointers I find that everything is fine for small matrix dimensions, for example 10x10. As I start increasing the matrix dimension (more than 100x100) I reach a point where the pointers that I get from MathcadArrayAllocate are confused, that is, CArray->hReal and CArray->hImag are different addresses but these two addresses point to the same memory address (the first element of the array). Consequently, the real and imaginary parts overwrite each other. For even larger matrix dimensions (say 640 x 480) one of the allocated pointers (hReal) points to an address that points to a null pointer, causing a GP which is reported in Mathcad. I've signaled this to Mathsoft when 2001 came out and even provided an example DLL but so far I haven't had any feedback. This has forced me to revert to other tools to perform certain tasks.
Regarding your specific problem a possible solution is to use GlobalAlloc and GlobalFree inside your function. This works fine to allocate temporary memory buffers at function scope.
FYI, I've also been able to setup a DLL that possesses global variables outside of the scope of the functions called by Mathcad. Hence, it is possible for one function to generate data, store them in a global buffer allocated using GlobalAlloc and return to Mathcad (without deallocating of course). I can then call other functions within the same DLL and they all have access to the data. A simple example is a function that loads a large data file in memory (I send the file path to the DLL using str2vec). Once done verifying the data and loading it into the global buffer, the function returns to Mathcad. I then call different functions to extract various parts of the data or process various subregions, etc. This has worked under Mathcad8, 2000 and 2001. For "cleanliness" I have a function that can deallocate the global buffer but I've found that since it resides in the DLL memory space, the buffer gets deallocated cleanly when Mathcad exits.
Hello Xavier, I read your post of September 28 and immediately recognized the likely cause my recent problems with DLL's crashing Mathcad. I have written a dozen successful dll's for Mathcad 8 on Windows NT using Visual C++ 5.0 and recently began using Mathcad 2001 on Windows 2000 Pro. Some of the same dll's now crash and new ones written with the most recent versions of the include and library files crash too. The problem is most common with large return array sizes. Just before I read your post I tried not allocating the imaginary portion of the array (because I am passing only real numbers): (code example follows) // allocate memory for the output .... if ( !MathcadArrayAllocate( Matrixout, 2000000 , 3, TRUE , FALSE ))
and it seemed to help. You wrote "As I start increasing the matrix dimension (more than 100x100) I reach a point where the pointers that I get from MathcadArrayAllocate are confused, that is, CArray->hReal and CArray->hImag are different addresses but these two addresses point to the same memory address (the first element of the array). Consequently, the real and imaginary parts overwrite each other." Have you found more info about the problem? Is my simple solution (no imaginary part) reliable? Does Mathcad 2001i have this problem? Did Mathcad 2000 have this problem? I noticed your solution of using "GlobalAlloc" and I wonder if you could send me some sample code that uses this solution.
Yes, the problem appeared with 2001 and was fixed in 2001i. I had functions in 2001 that used to return a complex array. I wrote new functions that returned only a real array, but that didn't work for larger arrays (for example 640 x 480). I also observed at the time that some of my functions would sometimes work, that is, the first call in Mathcad would return a memory allocation error but hitting F9 would get the call the succeed. I then tried to trap this condition from within the DLL but that also failed (successive memory allocation attempts until the array pointer was valid). So I pretty much gave up on 2001 - Mathsoft acknowledged the problem and promised it would get fixed, which happened in 2001i. I can only recommend you do that.
Regarding GlobalAlloc: I'm not sure if you're considering this as a possible solution to your memory allocation problem. As far as I know, you can't use GlobalAlloc to create a Mathcad return array. I use this only to allocate a permanent memory buffer in the DLL address space. One of my DLL functions creates this buffer and fills it with data from a file. Other DLL functions can then perform various tasks on the data without having to reload the file. The benefit for me is that a 100MB binary file where each sample is a byte requires a 100MB memory buffer, whereas if the file was loaded as a Mathcad array a 800MB chunk of memory would be required...
Xavier and Robert (and Mr. Guy Beadie if he still reads these),
Many thanks for your posts. I am so relieved to be able to contact people who have a better idea what's going on (than Mathcad support).
Regarding the memory allocation problem, I seem to be having pretty good luck with only allocating the real part. Even with arrays that are 3 x 1 million. Perhaps Mathsoft fixed it in SR2 of 2001. I have a call into them and perhaps they will clear this up. My company presently does not supply 2001i but maybe this will be a sufficient reason to change that.
Let me mention one other thing I noticed with dll's in case people are seeing other artifacts with 2001. A new "official" string variable was introduced that allows one to pass a string variable into a dll. It works just fine as long as all variables passed into the dll are string, but it doesn't work with mixed variable types where, for example, one of the function variables might be a complex scalar. The response I got from Mathsoft on this issue is that the string type is not supported even though it is in the official release package. When I need to pass strings into a dll, I do it through an element of a matrix. This is also not officially supported, but I haven't had any problems.
If people need to pass string variables back out of the dll, the only way I know to do that is to use the scripted component interface. This interface, however, has the problem that it won't allow passing complex variables (it's a bug, not a design), and it's not nearly as handy as the dll for basic functions.
For exchanging strings back and forth I use the brute force approach: (1) From Mathcad: use str2num(...) to create an array that is passed to the DLL. The DLL then needs to extract the ASCII characters and rebuild the string.
(2) From the DLL: generate an array with MathcadAllocate and fill it with the ASCII codes of the string. Use num2str in Mathcad.
All those operations are supported since we're only passing regular arrays.
For what it's worth, I was able to use GlobalAlloc() successfully for all my DLL memory allocation in Mathcad 2001.
Once I substituted GlobalAlloc() for MathcadAllocate(), everything worked fine. I never totaled up the dynamically allocated memory, but some of the return arrays of real doubles were as large as 6 x 32000 and 56 x 1000.
On 9/12/2002 8:20:00 PM, Anonymous wrote: >Does anyone know who frees the >GlobalAllocate() memory when >Mathcad is done with it? I've >never checked. > >Robert
I think the whole point of using MathcadAllocate instead of malloc or GlobalAlloc is that Mathcad takes care of freeing memory buffers when it's done with them. When you use malloc or GlobalAlloc your DLL is in charge of doing the garbage collection, Mathcad won't do it. In practice, if you don't take care of it the memory will remain allocated until you exit Mathcad, which is no big deal for small buffers. In my case I've added specific functions to my DLLs to free up the larger buffers that are statically allocated in the DLL address space.
Guy (and Xavier and Robert), I haven't tried GlobalAlloc() yet, but I have now had a week of experience with simply not allocating the imaginary part when using MathcadAllocate(). (I set the second true/false argument to false). Two of my dlls which formerly catastrophically crashed with return arrays as small as 100,000 x 1 (complex values) now work with arrays as large as 1,000,000 x 3 (real values only). And I can make repeated calls to the dll and use the F9 key repeatedly. I noticed in another thread that someone reported that inserted components also seem to have problems returning complex numbers and the reported solution there was to simply not do it (and return the imaginary part in another portion of a real array if the imaginary part is really required).
I'm sorry, I realize that this previous post of mine is not very accurate. A typical DLL function you write should by default use MathcadAllocate to create memory buffers and MathcadFree to destroy them just before exiting the function. I believe that MathcadAllocate is nothing else than a wrapper that allows Mathcad to keep track of memory buffers allocated by the user. Hence, if your function does not call MathcadFree upon exiting, Mathcad is still in a position to free these user buffers in a garbage collection phase. However, if you use malloc or GlobalAlloc, Mathcad is not aware of the existence of user buffers and won't be able to release them if the function doesn't. Now, GlobalAlloc becomes useful in the case where you want to allocate memory having a lifetime longer than the lifetime of a DLL function call. In that case you can create a bunch of functions in your DLL that know how to access this persistent memory buffer. It's in this case that it may be useful to write a function that specifically releases the global buffer. If you never call such a function this memory is released once you exit Mathcad and its address space is freed. Hope this makes more sense... Xavier
Thanks for the clarification. I was wondering if Mathcad somehow automatically grabbed ownership of the allocated memory like some clipboard functions would. I've toyed with the idea of using global allocation to return strings to Mathcad and having the dll keep internal track of the memory allocation, but I haven't found a very pretty solution.
In response to freeing memory after using GlobalAlloc(), I was always careful to deallocate any dynamic memory myself. I balanced every call to allocate an array with a call to free the array when I was done with it.
I should point out that I still used MathcadArrayAllocate() for the final return array, but I always used real arrays. My problems originally stemmed from using MathcadArrayAllocate(), even with real numbers, to define arrays used during calculations. After a certain number of allocations, the arrays started writing over themselves.
As an added safety measure I also made sure that the return array, FinalVals, was allocated at the end of all the calculations. The only steps I perform after the call to MathcadArrayAllocate() are inserting the proper values into FinalVals and deallocating the remaining dynamic arrays, used to hold the final values during computation.
Sticking to those guidelines kept me from running into any further trouble with the memory problem.
That's interesting. I usually allocate the return array first (with MathcadArrayAllocate) and then allocate the required local memory buffers using MathcadAllocate or malloc. I can't remember if I tried allocating the return array at the end of the processing when I was having these problems in 2001. Xavier
On 9/13/2002 11:31:00 AM, xcolonna wrote: >I'm sorry, I realize that this >previous post of mine is not >very accurate. >A typical DLL function you >write should by default use >MathcadAllocate to create >memory buffers and MathcadFree >to destroy them just before >exiting the function. I >believe that MathcadAllocate >is nothing else than a wrapper >that allows Mathcad to keep >track of memory buffers >allocated by the user. Hence, >if your function does not call >MathcadFree upon exiting, >Mathcad is still in a position >to free these user buffers in >a garbage collection phase. >However, if you use malloc or >GlobalAlloc, Mathcad is not >aware of the existence of user >buffers and won't be able to >release them if the function >doesn't. >Now, GlobalAlloc becomes >useful in the case where you >want to allocate memory having >a lifetime longer than the >lifetime of a DLL function >call. In that case you can >create a bunch of functions in >your DLL that know how to >access this persistent memory >buffer. It's in this case that >it may be useful to write a >function that specifically >releases the global buffer. If >you never call such a function >this memory is released once >you exit Mathcad and its >address space is freed. >Hope this makes more sense... >Xavier
This is very useful if you need to load large amount of data that is reused with every call. I have a question - how do you address the memory that was allocated in previous call. Isn't the created by GlobalAlloc pointer dynamic - you would need to access its actual value to find where the memory block is?
Could you post a C++ example of doing this?
You have these nice DLLs WriteGlobal and ReadGlobal, which I guess would be a good example (except they dont have a call that clears the memory).
On 4/1/2010 11:31:24 AM, evgen wrote: >I have a question - how do you address >the memory that was allocated in >previous call. >Isn't the created by GlobalAlloc pointer >dynamic - you would need to access its >actual value to find where the memory >block is?
If you are asking about how to use memory allocated by GlobalAlloc, it follows the normal rules of C or C++.
If you are asking about creating memory space in the first call to the DLL that stays static, and can be accessed in another call to the DLL I cannot help you.
I only defined temporary arrays - arrays that were deleted when I finished executing the DLL code.
I understand now. I guess I never ran into these sorts of problems because I only use the Mathcad memory allocation functions to return data to Mathcad. My normal language is c++ which I think uses malloc in the new function. I started using the Mathcad memory functions in my functions once upon a time, but I got away from that because I quickly found that c++ handled complex numbers better as well as enabling me to use arrays that cleaned up after themselves.