rosasurfer/mt4-expander

Could I use this expander for opening chart windows?

redame opened this issue · 29 comments

could i use i the libary to develop my dll to control mt4 ?

could i open new chart out of mt4 with dll or other exe?

some question about symbols.sel in mt4
how could I update marketwatch info with update and write market info to sel file in realtime?

First, thanks for your interest. :-)

No, this library is the DLL working with/controlling MT4. It's the C/C++ part of a framework. The MQL part using the DLL is a separate Github project "mt4-mql".
Main purposes:

  • Features not provided or not possible with MQL: e.g. communication between MQL modules; the indicator called by iCustom() can send back error codes to the calling module; the calling module can react; same for EA <-> MQL library communication.
  • Pointer functionality for MQL4: you can modify memory (arrays) directly
  • Struct handling in MQL4; you don't need many of the new features in MT5/MQL5. Everything is doable in MQL4.
  • Timer related functionality (self-updating offline charts). Correct indicator calculations in offline charts (IndicatorCounted() does not work there).
  • Creation and charting of synthetic instruments: you can attach your own pricefeed to such a synthetic instrument and it updates in near-realtime (something MetaQuotes just started working on in MT5).
  • Charting of account balances and real equity charts or of equity charts resulting from backtests. For an example see the following post (the chart is generated by this framework):
    https://strategyquant.com/forum/topic/can-quantanalyzer-show-helpfull-results-think-again/
  • And of course working around the almost countless bugs and limitations of MQL. Ever wondered why immediately after a test in the next one the order functions still return ticket data from the previous test? Because the EA is not completely unloaded from memory and still contains state from the previous test. This even if you change the test's symbol or timeframe.

Modifying symbols.sel is not giving you anything useful. If you know the FIX protocol, the .sel file just tracks your subscribed symbols and the data of the last tick. MetaQuotes just calles it "selected" instead of subscribed.
You cannot "merge" two feeds, i.e. injecting your own ticks in an already attached feed. That's only possible for the MT manager plugin owned by your broker. What you can do is turning an offline chart (which you feed your own prices) into a seemingly regular one (selfupdating with timeframe switching). It's a hack but it's stable.

many thanlk for your reply, and
Now I can update chart in realtime with write the hst file via other program via MdiClient update for metaquote windows.
My purpose is to connect china stock market to mt4 and update the chart( have done ) and see the updating makret info in marketwatch,(still finding way)
Or I can creat a market info window in my program then click and open chart in mt4

I don't have an updating marketwatch window. I have a ChartInfos indicator which can show big price and spread if I want. Inter-process communication works via QuickChannel. You can update the chart directly.
btw: For my own instruments I always use old MT4 versions which MetaQuotes cannot modify/restrict anymore. I just want to be independent.

At the moment I'm focusing on integration of test analysis into the framework. The DLL collects data and calculates test statistics which an EA might use for self-recalibration. UI for this is in the works in the mt4-tools project which is at http://xtrade.rosasurfer.com/

Example of synthetic online charts:
On the left a connected terminal which calculates price and spread for an non-connected terminal on the right. The green light at top right signals that the chart indeed is connected, for the terminal it's a regular built-in symbol as can be seen in the symbols window.

synthetics

I have already arrived your website before via google search to find the expander usage.
at the moment Out team are developing a mt4 data feed plugin to view china stock market,, we have finished symbols.raw and symgroups.raw modified with china symbol and chart update with hst rewrite. both of these we don't need any mql4 and mq4 in mt4. the only thing is updating the marketwatch still on the way, we update the chart via AfxFrameOrView and MdiClient with winddows api. updateing marketwatch windows in our plugin windows is easy to do, but i can't open mt4 chart from our plugin windows by click the symbol name.
Do have any idea for this, or can i make some dll with the plugin to call mq4 function to open new chart?

Ah, I understand. Could something be wrong with your symbols.raw? I have no problems with opening synthetic charts from MarketWatch the regular way. I just don't have prices there but I don't need them.

MarketWatch

I can open chart form marketwatch that isn't update,
there are two way for me to implenment
1, I means could i open chart from outside program?
2,You have told me i cann;t update marketwatch with override the data in sel, Is there any way in menory to terminal for update marketwatch? like update chart from hst without any mq4

My plugin isnot an another mt4 terminal,

  1. "...I means could i open chart from outside program?..."
    Just send the proper message. If from outside, hook a window procedure, send a user message and translate it to whatever you want.

  2. You mean which messsage to send to update the chart from hst? That depends on your environment. My hack is just an inhouse solution to get updating charts, it's not meant for distribution. I share if you are interested but you cannot give something like this to clients. It's ugly.

The code for updating synthetic charts is in
https://github.com/rosasurfer/mt4-expander/blob/7a2de86/src/lib/timer.cpp
It's just a regular message depending on the chart type the timer is running on. Plus I can configure from MQL to only update if the chart is somewhat visible because updating from .hst means to re-read the full .hst file. It will be very resource cosuming if your chart has 100k bars. Imagine you re-read those 100k bars on each tick.

Now the trick is to make the terminal handle your "online" chart as it would handle an offline chart. On an online charts WM_REFRESH means "check the broker's feed for updates". On an offline chart it means "re-read the history".

To do this you have to open the symbol first as an offline chart and after you open another chart of the same symbol as on online chart. If done in this order the terminal will handle the online chart like an offline chart. You can now close the first offline chart and the online chart will continue to behave like an offline chart.

This only works on a multi-core machine, probably because the window management takes place in a different thread. In a VPS with a single core it doesn't work. Timing issues...

1,do you have any code to implement this way? in mql4 to open chart via chartopen(symbol period)
in my pliug there is also a symbol list,when i click one symbol in it and will trigger this symbol chart open
2 I have finished the chart update works. MetaQuotes::MetaTrader::4.00 window and replace the handle MDIClient and send message.
by the wayI am newer in github, how could i access you rosasurfer/mt4-mql

I searched MetaQuotes::MetaTrader::4.00 in github so I found your code and think we are in same way

"...do you have any code to implement this way? in mql4 to open chart via chartopen(symbol period)..."

I don't have such a code because I don't need it. But it's easy to implement. Inspect the Windows message sent from the context menue in the MarketWatch window. The tricky is to find the symbol id which should be the same as in symbols.raw. Beware, there are two ids. It should be SYMBOL.id at offset 120, not SYMBOL.arrayKey at 116.

The symbol definition I referr to is
https://github.com/rosasurfer/mt4-expander/blob/7a2de86/struct/mt4/Symbol.h

I use WinSpector for finding/filtering messages, with it you can filter nicely.

(2) There could be second way. My guess is the built-in function ChartOpen() does exactly the same. At least this is my experience with the code in MT4. So you should be able to sniff the right message by following a ChartOpen() call with WinSpector.

(3) Then there is a third way which is the main menu File->NewChart command. The message sent from there will be a command id (not a regular message) and for the dynamic menu parts it will be dynamically generated. You can inspect the menu and extract the ids at runtime with a tool or directly from your application.

For (2) I checked (but didn't test). The docs even explicitely say so:
https://docs.mql4.com/chart_operations/chartopen

You can find the message/command with WinSpector and send it manually like this:

PostMessage(hWnd, WM_*****, lParam1, lParam2));

`
WM_SETTEXT

03:09:30.0863

Text: Open chart window



WM_PAINT

03:09:30.0863

wParam: 0x00000000
lParam: 0x00000000



WM_ERASEBKGND

03:09:30.0863

		<parameter>HDC: 0x580143e3</parameter>
	</parameters>
</message>
<message return-value="1">
	<name>WM_ERASEBKGND</name>
	<sent />
	<time>03:09:30.0863</time>
	<parameters>
		<parameter>Processed: True</parameter>
	</parameters>
</message>
<message>
	<name>WM_GETFONT</name>
	<sent />
	<time>03:09:30.0864</time>
	<parameters>
		<parameter>wParam: 0x00000000</parameter>
		<parameter>lParam: 0x00000000</parameter>
	</parameters>
</message>
<message return-value="1">
	<name>WM_GETFONT</name>
	<sent />
	<time>03:09:30.0864</time>
	<parameters>
		<parameter>lfHeight: -12</parameter>
		<parameter>lfWidth: 0</parameter>
		<parameter>lfEscapement: 0</parameter>
		<parameter>lfOrientation: 0</parameter>
		<parameter>lfWeight: 400</parameter>
		<parameter>lfItalic: 0</parameter>
		<parameter>lfUnderline: 0</parameter>
		<parameter>lfStrikeOut: 0</parameter>
		<parameter>lfCharSet: 1</parameter>
		<parameter>lfOutPrecision: 0</parameter>
		<parameter>lfClipPrecision: 0</parameter>
		<parameter>lfQuality: 0</parameter>
		<parameter>lfPitchAndFamily: 0</parameter>
		<parameter>lfFaceName: Microsoft YaHei UI</parameter>
	</parameters>
</message>
<message return-value="1">
	<name>WM_PAINT</name>
	<sent />
	<time>03:09:30.0864</time>
	<parameters>
		<parameter>Return: 0x00000000</parameter>
	</parameters>
</message>
<message>
	<name>WM_GETTEXTLENGTH</name>
	<sent />
	<time>03:09:30.0871</time>
	<parameters>
		<parameter>wParam: 0x00000000</parameter>
		<parameter>lParam: 0x00000000</parameter>
	</parameters>
</message>
<message return-value="1">
	<name>WM_GETTEXTLENGTH</name>
	<sent />
	<time>03:09:30.0871</time>
	<parameters>
		<parameter>Text length: 18</parameter>
	</parameters>
</message>
<message>
	<name>WM_GETTEXT</name>
	<sent />
	<time>03:09:30.0871</time>
	<parameters>
		<parameter>Text buffer pointer: 0x0065c0a4</parameter>
		<parameter>Text buffer length: 19</parameter>
	</parameters>
</message>`

I have tested sniffer with wininspector and get the message as above
how about next
where can i find the symbol which I choose

</message>
<message>
	<name>WM_COMMAND</name>
	<posted />
	<time>02:59:40.0213</time>
	<parameters>
		<parameter>Code: 0</parameter>
		<parameter>Control ID: 33160</parameter>
		<parameter>Control HWND: 0x00000000</parameter>
	</parameters>
</message>
<message>
	<name>WM_COMMAND</name>
	<posted />
	<time>02:59:40.0213</time>
	<parameters>
		<parameter>Code: 0</parameter>
		<parameter>Control ID: 33160</parameter>
		<parameter>Control HWND: 0x00000000</parameter>
	</parameters>
</message>

I also found command as above, But i still cann't find the symbol

At the moment I'm quite busy so I will come back to this later. You need to filter the messages you are not interested in, that's everything related to drawing, mouse movement, DC etc. There are hundreds of messages, it's overkill without filtering.
As for the second dump: You can see the control id (33160), that's the dynamically created one I mentioned above. Each id belongs to one symbol. You need to walk through your menu, read the text (that's the symbol) and resolve the id belonging to it. In fact, that's similar to how Windows does it by itself. If you don't have a C/Win32API programming background you will need to look for somebody who can do that for you. MQL is not C and far from enough.

If I find some time I can do it, unfortunately not at the moment. Don't be discouraged, give it a try.

now,As you mentioned ,I cann't open the updating chart via marketwatch as online, So I have to find another method to open offline chart out of mt4 and I have know to open the offline chart select window form WM_COMMAND, 33053, and how could i find the specific file which I wanna open from my plugin only click

Digged into the messages on ChartOpen() and found the ids.

First the screenshot of WinSpector with filtered messages in the moment a chart is opened from the main file menu. I started with no filter at all and filtered one message by another until only WM_COMMAND and registered messages where left:
Windows messages on ChartOpen

As you can see after opening the "File -> New Chart" submenu (WM_COMMAND) the actual chart opening is triggered by an application registered message. It passes a command (wParam=0x0033) and an id (lParam=...) pointing to the symbol in the MarketWatch window:

lParam=0x8ca0 for the first symbol
lParam=0x8ca1 for the second symbol
lParam=0x8ca2 for the third symbol and so on.

The main menu "File -> New Chart" is populated with symbols in a very similar way; the menu order always reflects the symbol order in the MarketWatch window.

If you want to open a chart window you first have to inspect the MarketWatch control and read the selected symbols (you can only open charts for symbols available in MarketWatch). You read the position of the desired symbol starting with 0 (zero) and translate it to a chart id by adding 36000 (hex 0x8ca0 = decimal 36000). The resulting value position + 36000 you pass as the second parameter of the registered message (lParam).
Translated symbol ids in MarketWatch

In MQL it could for example look like the following script:

/**
 * Example script opening a chart window.
 * Compiled with MT4 build 226, tested with MT4 build 225 to 1090.
 */
#property show_inputs

extern int Symbol_Id = 36000;     // translated symbol id starting at 36000 for the top-most symbol

#import "Expander.dll"
   int  GetApplicationWindow();
   int  MT4InternalMsg();
#import "user32.dll"
   bool PostMessageA(int hWnd, int msg, int wParam, int lParam);
#import

#define MT4_OPEN_CHART           51       // hex 0x0033 = decimal 51
#define ERR_USER_ERROR_FIRST  65536

/**
 * Main function
 *
 * @return int - error status
 */
int start() {
   int hWnd = GetApplicationWindow();
   if (!hWnd) return(ERR_USER_ERROR_FIRST);

   PostMessageA(hWnd, MT4InternalMsg(), MT4_OPEN_CHART, Symbol_Id);

   return(0);
}

You can inspect the MarketWatch control with the standard Win32 API as documented in MSDN. The required Expander.dll can be found in the libraries folder of the mt4-mql project which I gave you access to (mql4/libraries/Expander.dll). Or you build it by yourself from this project.

many thanks for your help and code, Now If I wanna open updating chart in mt4, I have to open offline chart as your mention, So I found the SendMessageA(hwnd, WM_COMMAND, 33053, 0) to trigger "open offline chart" window, And I think the command to open offline chart in this window,

Could you help me to find the command and position to open chart from this window?

WM_COMMAND

02:30:41.0727

Code: BN_CLICKED
Control ID: 1
Control HWND: 0x00630802


I found this command when I open offline chart via ketboard.

BTW I found an artical https://www.mql5.com/en/articles/2297#chapter2_1
That mean I can open offline chart via ChartOffID = ChartOpen ( Symbol (), ExtOffPeriod);
This code opens an offline graph with the period " ExtOffPeriod " - the truth before this is to create a history file.

Yes, this should work. I thought about something similar. However, in general the article's approach is way to complicated, the source code is bloated and the API not abstract. In fact there is no API.

A better approach would be mql4/libraries/history.mq4, the header is in mql4/include/history.mqh. It's not translated but you should get the important parts.

/**
 * Funktionen zur Verwaltung von Historydateien.
 *
 *  • Alte MetaTrader-Versionen (Format 400) löschen beim Beenden neue Historydateien, wenn sie auf sie zugegriffen haben.
 *  • Neue MetaTrader-Versionen (Format 401) konvertieren beim Beenden alte Historydateien ins neue Format, wenn sie auf sie zugegriffen haben.
 */
#import "history.ex4"

   // Symbol-Management
   int  CreateSymbol(string name, string description, string group, int digits, string baseCurrency, string marginCurrency, string serverName="");

   // HistorySet-Management
   int  HistorySet.Create (string symbol, string description, int digits, int format, string server="");
   int  HistorySet.Get    (string symbol, string server="");
   bool HistorySet.Close  (int hSet);
   bool HistorySet.AddTick(int hSet, datetime time, double value, int flags=NULL);

   // HistoryFile-Management
   int  HistoryFile.Open     (string symbol, int timeframe, string description, int digits, int format, int mode, string server="");
   bool HistoryFile.Close    (int hFile);
   int  HistoryFile.FindBar  (int hFile, datetime time, bool lpBarExists[]);
   bool HistoryFile.ReadBar  (int hFile, int offset, double bar[]);
   bool HistoryFile.WriteBar (int hFile, int offset, double bar[], int flags=NULL);
   bool HistoryFile.UpdateBar(int hFile, int offset, double value);
   bool HistoryFile.InsertBar(int hFile, int offset, double bar[], int flags=NULL);
   bool HistoryFile.MoveBars (int hFile, int fromOffset, int destOffset);
   bool HistoryFile.AddTick  (int hFile, datetime time, double value, int flags=NULL);
#import

With something like this you don't work with single history files but with complete history sets. If you add a tick its added to all timeframes, not only to one.

Now something are clear, They are 1)we can open online charts with winapi of you code via my plugin, And the chart can't be updated as online mode. but it can change the timeframe.
2)As you mention we can simulate the tick in offline chart which cann't change timeframe
3) I can open an online chart after offline chart, the online chart will be update like offline, even I close the offline chart.(I don;t know how about when I change timeframe of this online chart)

On you last reply, mql4/include/history.mqh Do you means If I write complete histry sets, I can switch the timeframe with updating onlime charts that depond on re-read hst? Plz tell me more detail about it

I got some thought from https://www.mql5.com/en/articles/2297#chapter1 this aticle
this indicator can open offline chart like online chart(must be applied in M1 chart), the reason is period in hst header, When I change M2=2 // period M2 to M2=5, // period M2 It genarate a M5 online chart instead of M2 offline chart?
I mean could I change the HST head to implement both update and timeframe switch?

best regards

  • You can always switch timeframes if the chart was opened as an online chart (from MarketWatch or from File->New Chart).

  • Each timeframe will show the price history if it is available (e.g. if a history file exists) no matter how you opened a chart. It also doesn't matter how the history file was created (i.e. manually). If no history file exists switching still works but the chart window will show the message "Waiting for update...".

  • For the following to work disconnect your terminal. If you first open nine offline charts for the nine timeframes these nine offline charts will react to WM_REFRESH and re-read the history file. If after these nine charts you open one online chart of the same symbol the online chart will react to WM_REFRESH and re-read the history files the same as an offline chart if on a multi-core machine. You will have a chart with switchable timeframes that looks like an online chart but acts like an offline chart. This behaviour will continue if you close all of the nine offline charts you opened in the first place.

  • Opening nine offline charts plus one online chart can be automated by using a chart profile with nine offline charts and one online chart (in this exact order). Closing the nine offline charts after such a profile is loaded can be automated by custom code (MQL or DLL) which iterates over the chart windows and closes only the offline charts. When you switch profiles the terminal will delete the files for the offline charts (because you closed them). To prevent the terminal from doing so protect the files by setting their ttributes to read-only.

In the above description when I say "offline charts" I mean offline charts on built-in broker symbols, not offline charts on newly created history files on non-broker symbols (like the MQL.com articles do which you mentioned). If your terminal is disconnected you can for example overwrite all nine history files of EURUSD with your Chinese index data and create an updating chart the way I described above. Or you can create your own brand-new symbol and add it to the existing broker symbols by creating a new entry in symbols.raw. This is what I'm doing in the charts I showed above. I added new symbols for indexes, account equity and other statistics. See the function CreateSymbol().
Again, timeframe-switchable and self-updating synthetic charts will only work with symbols in symbols.raw.

In fact your terminal "thinks" your new symbols are regular broker symbols. If done correctly the terminal cannot see the difference.