Friday, January 25, 2008
How to detect Mobile device hitting your Web Server in PHP
by Andrea Trasatti
http://wurfl.sourceforge.net/index.php
PHP is a great platform for WAP development. Thousands of developers worldwide love PHP for its performance and for the semplicity of its model.
It should come as no surprise that some PHP developer quickly built the tools to tap the WURFL power from PHP.
One easy way to play with the WURFL, is to use what I call the "WURFL PHP Library". The package includes the library and some support files such as a readme (I STRONGLY SUGGEST EVERYONE TO READ IT), check_wurfl.php that will let you quickly read all the capabilities of a selected device and update_cache.php which will be better described later.
* wurfl_parser.php: parse the xml and put it in an Array and the other to work with the data collected. Before starting, make sure you have compiled PHP properly ( http://www.php.net/manual/en/ref.xml.php ), wurfl_parser.php uses the basic XML functions implemented compiling with expat.
* wurfl_class.php: this script lets you access the data in the wurfl array (see previous bullet) in an object-oriented fashion.
* wurfl_config.php: with the growing number of possible configurations in the PHP library, I am now introducing a single file for configuration. This should hopefully make it clearer where you need to configure the scripts and maybe also make it easier to integrate it with your central configuration files (if your application has any)
wurfl_parser.php
wurfl_parser.php: This is a VERY simple XML parser that reads WURFL and puts all the needed data into an array. Considering the time the parser was taking to parse the XML, I thought that using some kind of caching mechanism was probably a good idea. This will be discussed more later.
The generated Array is an associative array that looks like this:
$wurfl["devices"]["ericsson_generic"]["fall_back"]="generic";
The first key is "devices". Other possible values such as authors and contributors are probably not so interesting to you.
The second key is the UNIQUE ID assigned to each user agent. All other keys are related to the attributes or groups and capabilities of the device.
Of course, this may seem not so flexible in practice, since devices tell us their user-agent strings, not their WURFL IDs.
This is solved thanks to an additional array called $wurfl_agents, which is simply an associative array of user agents and the relative unique id. This array makes your life much simpler (and your search much faster). $wurfl_agents is cached too.
Let's look at a concrete example. Someone visits you site with, say, a Siemens S45 (user agent: "SIE-S45/24 UP.Browser/5.0"). In order to find the device ID, you would normally be required to cycle most of the $wurfl array and retrieve the ID, before you can look up the actual capabilities.
Thanks to $wurfl_agent, this is as simple as:
if ( in_array($id, $wurfl_agents) ) {
echo "The ID $id is known
\n";
}
Example to search an ID knowing the user agent:
$wurfl_user_agents = array_keys($wurfl_agents);
while ( $x = each($wurfl_user_agents) ) {
if ( $x[1] == $user_agent ) {
return $wurfl_agents($x[1]);
}
}
wurfl_class.php
wurfl_class.php was created to make our life simpler. Once we had the data ready in an array, we needed an easy way to access it and some methods to manipulate it. By looking closely at what had been done with the Java API, I reproduced many of those useful methods in PHP.
The class loads the parser automatically. To load the parser a defined must be set appropriately in the config file.
The class is initialized calling the constructor, wurfl_class and passing two variables that may be empty. The former is the full XML parsed (like the parser does) and the latter is the array of user agents and id's as generated by the parser. Pass two empty variables (or nothing) if you want them to be filled (when needed) or the real values if you already have them. The class will check the values and the cache files (if enabled) and decide what to do. Also check your configuration, because the behaviour will change.
Use the public method GetDeviceCapabilitiesFromAgent() passing the user agent to make the class search for the best fit and fill the object's properties. Once again, what the class does depends on your configuration, if you enabled cache files and so on.
Once you have instantiated the object and passed a user agent you may use all the class' methods.
Here is a list of properties:
$wurfl_class->_wurfl is the WURFL array (all of it)
$wurfl_class->_wurfl_agents is the associative array made of the user agents
and unique id's
$wurfl_class->user_agent the visitor's user agent
$wurfl_class->wurfl_agent the WURFL's best fitting user agent
$wurfl_class->id the corresponding id
$wurfl_class->GUI true if the device supports Openwave's GUI
extensions
$wurfl_class->browser_is_wap true if the device is WAP capable. It is here
only for legacy support, you should use the
is_wireless_device capability from WURFL.
browser_is_wap now has the same value as the
capability, when found. If you want to take full
advantage of this capability you should download
the web patch from the WURFL site or CVS!
$wurfl_class->capabilities the array of device's capabilities
Note: PHP (up to version 4.3) does not have any distinction between private and public methods. In the wurfl_class implementation, I named all private methods with a leading underscore to distinguish them from public methods without the underscore. If you are interested in knowing the details, just open the class, there are beautiful JAVADOC-like comments for each variable and method.
These libraries don't require register_globals anymore (from version 2 and up), but will not work with versions before 4.1.
wurfl_class($wurfl, $wurfl_agents) is the constructor. Built to work
best with the wurfl_parser.
GetDeviceCapabilitiesFromAgent($ua) given a user agent it will search WURFL
for the best fit
getDeviceCapability($capability) given a capability it will tell you the
value. Remember that capabilities might be string,
integer or boolean.
wurfl_config.php
The scope of this file is straight forward, modify it at your wish to configure the library to act as you like it best.
Please check the paragraph about caching for more info about cache files.
Here is a quick explanation of all the fields:
WURFL_CONFIG boolean, this is set to true by default, it's used as a simple
check to make sure the configuration was included. Add this to your
configuration files if you won't use wurfl_config.php, otherwise just
leave it as it is
DATADIR string, where all data is stored (wurfl.xml, cache file, logs, etc)
WURFL_FILE string, full path and filename of wurfl.xml
WURFL_PARSER_FILE string, full path and filename of wurfl_parser.php
WURFL_CLASS_FILE string, full path and filename of wurfl_class.php
WURFL_USE_CACHE boolean, true if you want to use a cache file (strongly
suggested). If only this parameter is set to true will be used
cache.php.
WURFL_USE_MULTICACHE boolean, true if you want to use Multicache files
instead of a single BIG cache file (cache.php)
MULTICACHE_DIR string, used only if you enabled Multicache, defines where
the cache files will be stored. WARNING: while cache.php will grow
in size but remain a single file, here the files will grow in
number. Expect more than 5000 tiny files.
MULTICACHE_SUFFIX string, suffix for the files generated using Multicache.
Useful if you use a caching system and don't want to load your
shared memory with a ton of tiny files.
CACHE_FILE string, with full path and filename of the cache file to use
(refreshed when a new WURFL is found, if WURFL_CACHE_AUTOUPDATE is
set to true)
WURFL_CACHE_AUTOUPDATE boolean, tells the class to automatically update the
cached files with a new XML is found. This is NOT suggested when
using MULICACHE because of the high number of files to be updated.
Race conditions are highly possible to happen. The use of
update_cache.php is strongly suggested for production
environments
WURFL_PATCH_FILE string, optional patch file for WURFL
WURFL_AGENT2ID_FILE string, used by wurfl_class.php. Used only when
WURFL_USE_CACHE is set to true
MAX_UA_CACHE integer, max number of user agents to store in
WURFL_AGENT2ID_FILE. Too high limits might give the opposite effect.
WURFL_LOG_FILE string, defines full path and filename for logging
LOG_LEVEL integer, desired logging level. Use the same constants as for PHP
logging
WURFL_AUTOLOAD boolean, true if you want the XML to be loaded at every
startup. If not, the XML will be loaded when needed.
Caching
Considering how slow PHP can be when parsing a big XML file, caching was a must.
Currently there are two caching systems. The older is activated when setting WURFL_USE_CACHE to true and uses DATADIR to store its files. The concept is quite simple, dump the array generated by the parsers in a big file, by default called cache.php (set by the define CACHE_FILE). In this file we also store the array called $wurfl_agents and a timestamp, useful to check if a new XML was deployed.
This system is very simple in its concept and worked well for quite some time. Considering the big size that cache.php was reaching it became a need (and in fact I always strongly suggested it) to use a caching system at PHP-level, such as Zend Accelerator, Turck cache, APC 2.0. Using such tools lets you store the cache file (cache.php) into shared memory and provides really good performances from the third hit on (first hit the XML is parsed: slow, second hit the cache is stored in shared memory: slow, third hit and on the cache is read from the shared memory: fast!).
The new caching system was dubbed "multicache" because instead of generating a single big cache file it generates 1 cache file for every device in WURFL. For this reason you will need to create a directory for this (or at least this is suggested) because the library will generate about 6000 tiny files when this feature is activated.
To activate the multicache system you will need to set WURFL_USE_CACHE to true.
CACHE_FILE will still be used, but the file will only contain the array $wurfl_agents and the timestamp.
Also set WURFL_USE_MULTICACHE to true, set the appropriate path for MULTICACHE_DIR, an absolute path is suggested. Don't forget the ending slash (for example '/tmp/cache/multicache/').
MULTICACHE_SUFFIX should be left unchanged in most cases. This will define the extension of the tiny files. You might want to set some strange extension if you want to avoid that those files are cached in shared memory by any PHP-cache, for example. Change it only if you know what you're doing!
The multicache system provides a MUCH faster data retrival, files are smaller and so it will take a way less time to read them. This will also mean a higher I/O on your system, consider returning to the older cache system if you have problems. Generating many tiny files also involves possible race conditions if a new XML is deployed and the library is configured to update automatically the cache. Read on for more info.
If WURFL_CACHE_AUTOUPDATE is set to true the library (specifically wurfl_class.php) will check the timestamp in cache.php against the file mtime of wurfl.xml. If the XML is newer than the cache it is reloaded. This is not suggested for production environments, if you have many concurrent hits you might have more than one process trying to refresh the same cache wasting a lot of resources. If you would like to avoid this you can set the automatic update to false and use the 'ad hoc' script called update_cache.php. This script was created to be called from command line (or a hidden URL if you'd like, but the command line is suggested when available) and force a cache update. This way the cache update will be prepared and the file will be changed at the very last second saving a lot of resources and having a single process do it. On sites with MANY hits you might also consider preparing the cache files on a separated system and moving them to the production server at once. There isn't a sccript to do this automatically at this time, but the new update_cache.php is already a step in that direction.
When setting WURFL_USE_CACHE to true you also enable another simple caching system (active both when using standard cache and multicache). When a user agent hits your site, this will most likely hit it again a few times. It would be stupid to search for all its capabilities at every second hit. For this reason we store the user agent and its capabilities in a file named after the value set in WURFL_AGENT2ID_FILE, this will make every second hit A LOT faster.
Storing all the user agents hitting your site will end up having a second full cache file and the benefit would reach zero. For this reason you can define a limit of user agents stored using MAX_UA_CACHE. The "perfect" value will change depending on your server's performances on the variety of user agents visiting your site and so on. A good number is between 30 and 50. I suggest you to start with 30 and maybe check the logs and see how often the cache is cleaned of the elder user agents and how often a user agent that is still visiting your site is cleaned and researched.
It's a direct consequence of the cache system that you will not need to read the entire XML and parse it every time you start the wurfl object. You may still want to force this for debug reasons for example. You can do this setting to true the define named WURFL_AUTOLOAD. If you are using any of the caching systems, I suggest you to disable this. If you're not using any cache, the XML will be loaded anyway, so just set this to true, if you'd like.
Logging
While logging is out of the scope of the WURFL PHP Libraries and I suggest you to integrate the libraries with your logging system (if you have any), a basic logging feature is included. This should work fine on Linux, Solaris and Windows.
Logging is done on a file as configured with WURFL_LOG_FILE. The log level is defined following the PHP contants and using the define named LOG_LEVEL. It used to be buggy in previous releases, check out how it has changed and it is now supposed to work properly. When set with the highest level the library might generate a log of logs.
Logs should anyway give you all the info you might need. This is a sample log when set at the highest detail level:
[LuckyTitan.local 327][constructor] Class Initiated
[LuckyTitan.local 327][GetDeviceCapabilitiesFromAgent] searching for SonyEricssonZ600/R601
[LuckyTitan.local 327][_cacheIsValid] cache file is outdated
[LuckyTitan.local 327][GetDeviceCapabilitiesFromAgent] cache enabled, WURFL is not loaded, now loading
[LuckyTitan.local 327][GetDeviceCapabilitiesFromAgent] loading WURFL from XML
[LuckyTitan.local 327][parse] No XML patch file defined
[LuckyTitan.local 327][GetDeviceCapabilitiesFromAgent] Searching in the agent database
[LuckyTitan.local 327][_GetFullCapabilities] searching for sonyericsson_z600_ver1_subr601
[LuckyTitan.local 327][_GetDeviceCapabilitiesFromId] reading id:sonyericsson_z600_ver1_subr601
[LuckyTitan.local 327][_GetDeviceCapabilitiesFromId] I have it in wurfl_agents cache, done
http://wurfl.sourceforge.net/index.php
PHP is a great platform for WAP development. Thousands of developers worldwide love PHP for its performance and for the semplicity of its model.
It should come as no surprise that some PHP developer quickly built the tools to tap the WURFL power from PHP.
One easy way to play with the WURFL, is to use what I call the "WURFL PHP Library". The package includes the library and some support files such as a readme (I STRONGLY SUGGEST EVERYONE TO READ IT), check_wurfl.php that will let you quickly read all the capabilities of a selected device and update_cache.php which will be better described later.
* wurfl_parser.php: parse the xml and put it in an Array and the other to work with the data collected. Before starting, make sure you have compiled PHP properly ( http://www.php.net/manual/en/ref.xml.php ), wurfl_parser.php uses the basic XML functions implemented compiling with expat.
* wurfl_class.php: this script lets you access the data in the wurfl array (see previous bullet) in an object-oriented fashion.
* wurfl_config.php: with the growing number of possible configurations in the PHP library, I am now introducing a single file for configuration. This should hopefully make it clearer where you need to configure the scripts and maybe also make it easier to integrate it with your central configuration files (if your application has any)
wurfl_parser.php
wurfl_parser.php: This is a VERY simple XML parser that reads WURFL and puts all the needed data into an array. Considering the time the parser was taking to parse the XML, I thought that using some kind of caching mechanism was probably a good idea. This will be discussed more later.
The generated Array is an associative array that looks like this:
$wurfl["devices"]["ericsson_generic"]["fall_back"]="generic";
The first key is "devices". Other possible values such as authors and contributors are probably not so interesting to you.
The second key is the UNIQUE ID assigned to each user agent. All other keys are related to the attributes or groups and capabilities of the device.
Of course, this may seem not so flexible in practice, since devices tell us their user-agent strings, not their WURFL IDs.
This is solved thanks to an additional array called $wurfl_agents, which is simply an associative array of user agents and the relative unique id. This array makes your life much simpler (and your search much faster). $wurfl_agents is cached too.
Let's look at a concrete example. Someone visits you site with, say, a Siemens S45 (user agent: "SIE-S45/24 UP.Browser/5.0"). In order to find the device ID, you would normally be required to cycle most of the $wurfl array and retrieve the ID, before you can look up the actual capabilities.
Thanks to $wurfl_agent, this is as simple as:
if ( in_array($id, $wurfl_agents) ) {
echo "The ID $id is known
\n";
}
Example to search an ID knowing the user agent:
$wurfl_user_agents = array_keys($wurfl_agents);
while ( $x = each($wurfl_user_agents) ) {
if ( $x[1] == $user_agent ) {
return $wurfl_agents($x[1]);
}
}
wurfl_class.php
wurfl_class.php was created to make our life simpler. Once we had the data ready in an array, we needed an easy way to access it and some methods to manipulate it. By looking closely at what had been done with the Java API, I reproduced many of those useful methods in PHP.
The class loads the parser automatically. To load the parser a defined must be set appropriately in the config file.
The class is initialized calling the constructor, wurfl_class and passing two variables that may be empty. The former is the full XML parsed (like the parser does) and the latter is the array of user agents and id's as generated by the parser. Pass two empty variables (or nothing) if you want them to be filled (when needed) or the real values if you already have them. The class will check the values and the cache files (if enabled) and decide what to do. Also check your configuration, because the behaviour will change.
Use the public method GetDeviceCapabilitiesFromAgent() passing the user agent to make the class search for the best fit and fill the object's properties. Once again, what the class does depends on your configuration, if you enabled cache files and so on.
Once you have instantiated the object and passed a user agent you may use all the class' methods.
Here is a list of properties:
$wurfl_class->_wurfl is the WURFL array (all of it)
$wurfl_class->_wurfl_agents is the associative array made of the user agents
and unique id's
$wurfl_class->user_agent the visitor's user agent
$wurfl_class->wurfl_agent the WURFL's best fitting user agent
$wurfl_class->id the corresponding id
$wurfl_class->GUI true if the device supports Openwave's GUI
extensions
$wurfl_class->browser_is_wap true if the device is WAP capable. It is here
only for legacy support, you should use the
is_wireless_device capability from WURFL.
browser_is_wap now has the same value as the
capability, when found. If you want to take full
advantage of this capability you should download
the web patch from the WURFL site or CVS!
$wurfl_class->capabilities the array of device's capabilities
Note: PHP (up to version 4.3) does not have any distinction between private and public methods. In the wurfl_class implementation, I named all private methods with a leading underscore to distinguish them from public methods without the underscore. If you are interested in knowing the details, just open the class, there are beautiful JAVADOC-like comments for each variable and method.
These libraries don't require register_globals anymore (from version 2 and up), but will not work with versions before 4.1.
wurfl_class($wurfl, $wurfl_agents) is the constructor. Built to work
best with the wurfl_parser.
GetDeviceCapabilitiesFromAgent($ua) given a user agent it will search WURFL
for the best fit
getDeviceCapability($capability) given a capability it will tell you the
value. Remember that capabilities might be string,
integer or boolean.
wurfl_config.php
The scope of this file is straight forward, modify it at your wish to configure the library to act as you like it best.
Please check the paragraph about caching for more info about cache files.
Here is a quick explanation of all the fields:
WURFL_CONFIG boolean, this is set to true by default, it's used as a simple
check to make sure the configuration was included. Add this to your
configuration files if you won't use wurfl_config.php, otherwise just
leave it as it is
DATADIR string, where all data is stored (wurfl.xml, cache file, logs, etc)
WURFL_FILE string, full path and filename of wurfl.xml
WURFL_PARSER_FILE string, full path and filename of wurfl_parser.php
WURFL_CLASS_FILE string, full path and filename of wurfl_class.php
WURFL_USE_CACHE boolean, true if you want to use a cache file (strongly
suggested). If only this parameter is set to true will be used
cache.php.
WURFL_USE_MULTICACHE boolean, true if you want to use Multicache files
instead of a single BIG cache file (cache.php)
MULTICACHE_DIR string, used only if you enabled Multicache, defines where
the cache files will be stored. WARNING: while cache.php will grow
in size but remain a single file, here the files will grow in
number. Expect more than 5000 tiny files.
MULTICACHE_SUFFIX string, suffix for the files generated using Multicache.
Useful if you use a caching system and don't want to load your
shared memory with a ton of tiny files.
CACHE_FILE string, with full path and filename of the cache file to use
(refreshed when a new WURFL is found, if WURFL_CACHE_AUTOUPDATE is
set to true)
WURFL_CACHE_AUTOUPDATE boolean, tells the class to automatically update the
cached files with a new XML is found. This is NOT suggested when
using MULICACHE because of the high number of files to be updated.
Race conditions are highly possible to happen. The use of
update_cache.php is strongly suggested for production
environments
WURFL_PATCH_FILE string, optional patch file for WURFL
WURFL_AGENT2ID_FILE string, used by wurfl_class.php. Used only when
WURFL_USE_CACHE is set to true
MAX_UA_CACHE integer, max number of user agents to store in
WURFL_AGENT2ID_FILE. Too high limits might give the opposite effect.
WURFL_LOG_FILE string, defines full path and filename for logging
LOG_LEVEL integer, desired logging level. Use the same constants as for PHP
logging
WURFL_AUTOLOAD boolean, true if you want the XML to be loaded at every
startup. If not, the XML will be loaded when needed.
Caching
Considering how slow PHP can be when parsing a big XML file, caching was a must.
Currently there are two caching systems. The older is activated when setting WURFL_USE_CACHE to true and uses DATADIR to store its files. The concept is quite simple, dump the array generated by the parsers in a big file, by default called cache.php (set by the define CACHE_FILE). In this file we also store the array called $wurfl_agents and a timestamp, useful to check if a new XML was deployed.
This system is very simple in its concept and worked well for quite some time. Considering the big size that cache.php was reaching it became a need (and in fact I always strongly suggested it) to use a caching system at PHP-level, such as Zend Accelerator, Turck cache, APC 2.0. Using such tools lets you store the cache file (cache.php) into shared memory and provides really good performances from the third hit on (first hit the XML is parsed: slow, second hit the cache is stored in shared memory: slow, third hit and on the cache is read from the shared memory: fast!).
The new caching system was dubbed "multicache" because instead of generating a single big cache file it generates 1 cache file for every device in WURFL. For this reason you will need to create a directory for this (or at least this is suggested) because the library will generate about 6000 tiny files when this feature is activated.
To activate the multicache system you will need to set WURFL_USE_CACHE to true.
CACHE_FILE will still be used, but the file will only contain the array $wurfl_agents and the timestamp.
Also set WURFL_USE_MULTICACHE to true, set the appropriate path for MULTICACHE_DIR, an absolute path is suggested. Don't forget the ending slash (for example '/tmp/cache/multicache/').
MULTICACHE_SUFFIX should be left unchanged in most cases. This will define the extension of the tiny files. You might want to set some strange extension if you want to avoid that those files are cached in shared memory by any PHP-cache, for example. Change it only if you know what you're doing!
The multicache system provides a MUCH faster data retrival, files are smaller and so it will take a way less time to read them. This will also mean a higher I/O on your system, consider returning to the older cache system if you have problems. Generating many tiny files also involves possible race conditions if a new XML is deployed and the library is configured to update automatically the cache. Read on for more info.
If WURFL_CACHE_AUTOUPDATE is set to true the library (specifically wurfl_class.php) will check the timestamp in cache.php against the file mtime of wurfl.xml. If the XML is newer than the cache it is reloaded. This is not suggested for production environments, if you have many concurrent hits you might have more than one process trying to refresh the same cache wasting a lot of resources. If you would like to avoid this you can set the automatic update to false and use the 'ad hoc' script called update_cache.php. This script was created to be called from command line (or a hidden URL if you'd like, but the command line is suggested when available) and force a cache update. This way the cache update will be prepared and the file will be changed at the very last second saving a lot of resources and having a single process do it. On sites with MANY hits you might also consider preparing the cache files on a separated system and moving them to the production server at once. There isn't a sccript to do this automatically at this time, but the new update_cache.php is already a step in that direction.
When setting WURFL_USE_CACHE to true you also enable another simple caching system (active both when using standard cache and multicache). When a user agent hits your site, this will most likely hit it again a few times. It would be stupid to search for all its capabilities at every second hit. For this reason we store the user agent and its capabilities in a file named after the value set in WURFL_AGENT2ID_FILE, this will make every second hit A LOT faster.
Storing all the user agents hitting your site will end up having a second full cache file and the benefit would reach zero. For this reason you can define a limit of user agents stored using MAX_UA_CACHE. The "perfect" value will change depending on your server's performances on the variety of user agents visiting your site and so on. A good number is between 30 and 50. I suggest you to start with 30 and maybe check the logs and see how often the cache is cleaned of the elder user agents and how often a user agent that is still visiting your site is cleaned and researched.
It's a direct consequence of the cache system that you will not need to read the entire XML and parse it every time you start the wurfl object. You may still want to force this for debug reasons for example. You can do this setting to true the define named WURFL_AUTOLOAD. If you are using any of the caching systems, I suggest you to disable this. If you're not using any cache, the XML will be loaded anyway, so just set this to true, if you'd like.
Logging
While logging is out of the scope of the WURFL PHP Libraries and I suggest you to integrate the libraries with your logging system (if you have any), a basic logging feature is included. This should work fine on Linux, Solaris and Windows.
Logging is done on a file as configured with WURFL_LOG_FILE. The log level is defined following the PHP contants and using the define named LOG_LEVEL. It used to be buggy in previous releases, check out how it has changed and it is now supposed to work properly. When set with the highest level the library might generate a log of logs.
Logs should anyway give you all the info you might need. This is a sample log when set at the highest detail level:
Wednesday, December 19, 2007
How to detect if Visual Studio 2005 SP1 is installed on Windows
If you don't need to be specific about a particular Visual Studio 2005 SKU, you can detect SP1 for a particular written language (for VSTS and TFS) or for a particular project language (for Express SKUs) using an older registry key under,
HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components\{PatchCode}
You can find the {PatchCode} in the .msp file. In Windows 2003 and earlier platforms, you'll see this as the Revision number property in the properties Summary tab in Windows Explorer. In all cases, you can query the Summary Information stream or open the .msp file in Orca, then click the View -> Summary Information menu item.
For the English Visual Studio 2005 Service Pack 1 for Standard, Professional, and Team editions (VS80sp1-KB926601-X86-ENU) an example of the registry key above follows.
HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components\{D93F9C7C-AB57-44C8-BAD6-1494674BCAF7}
You can also determine the service pack level of a more general, larger scope by reading a REG_DWORD registry value named SP from under,
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\[ProductFamily]\Servicing\8.0
…and for a specific language SKU using a sub-key,
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\[ProductFamily]\Servicing\8.0\[ProductEdition]\[ProductLanguage]
Product families are pretty broad in scope, and include for "Whidbey":
URT (.NET Framework, once known as the Universal Runtime)
VB (Microsoft Visual Basic 2005 Express)
VC (Microsoft Visual C++ 2005 Express)
VCS (Microsoft Visual C# 2005 Express)
VJS (Microsoft Visual J# 2005 Express)
VNS (Microsoft Visual Web Developer 2005 Express)
VS (Visual Studio 2005 Standard, Professional, Team Suite, etc.)
VSTF (Visual Studio 2005 Team Foundation Services)
An example using the English Visual Studio 2005 Team Suite Service Pack 1 of the registry keys above follows. Under each key you would find a registry value named SP set to 1.
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VS\Servicing\8.0
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VS\Servicing\8.0\VSTS\1033
Visual Studio 2005 Express SKUs will always use the ProductEdition of EXP, so you can combine the ProductFamily values documented above with EXP to form registry keys like the following, for Visual C# 2005 Express.
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VCS\Servicing\8.0
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VCS\Servicing\8.0\EXP\1033
HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components\{PatchCode}
You can find the {PatchCode} in the .msp file. In Windows 2003 and earlier platforms, you'll see this as the Revision number property in the properties Summary tab in Windows Explorer. In all cases, you can query the Summary Information stream or open the .msp file in Orca, then click the View -> Summary Information menu item.
For the English Visual Studio 2005 Service Pack 1 for Standard, Professional, and Team editions (VS80sp1-KB926601-X86-ENU) an example of the registry key above follows.
HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components\{D93F9C7C-AB57-44C8-BAD6-1494674BCAF7}
You can also determine the service pack level of a more general, larger scope by reading a REG_DWORD registry value named SP from under,
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\[ProductFamily]\Servicing\8.0
…and for a specific language SKU using a sub-key,
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\[ProductFamily]\Servicing\8.0\[ProductEdition]\[ProductLanguage]
Product families are pretty broad in scope, and include for "Whidbey":
URT (.NET Framework, once known as the Universal Runtime)
VB (Microsoft Visual Basic 2005 Express)
VC (Microsoft Visual C++ 2005 Express)
VCS (Microsoft Visual C# 2005 Express)
VJS (Microsoft Visual J# 2005 Express)
VNS (Microsoft Visual Web Developer 2005 Express)
VS (Visual Studio 2005 Standard, Professional, Team Suite, etc.)
VSTF (Visual Studio 2005 Team Foundation Services)
An example using the English Visual Studio 2005 Team Suite Service Pack 1 of the registry keys above follows. Under each key you would find a registry value named SP set to 1.
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VS\Servicing\8.0
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VS\Servicing\8.0\VSTS\1033
Visual Studio 2005 Express SKUs will always use the ProductEdition of EXP, so you can combine the ProductFamily values documented above with EXP to form registry keys like the following, for Visual C# 2005 Express.
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VCS\Servicing\8.0
HKEY_LOCAL_MACHINE\Software\Microsoft\DevDiv\VCS\Servicing\8.0\EXP\1033
Sunday, December 16, 2007
Locking header control in MFC to prevent the column to be resized
The way to prevent sizing is to eat this notification (without passing it to the header control), but as with many other Windows® messages and notifications, HDN_BEGINTRACK comes in two flavors: HDN_BEGINTRACKW (wide-character, Unicode) and HDN_BEGINTRACKA (ANSI). The "neuter" symbol is #defined to one or the other of these, based on the value of UNICODE defined in your project, as shown here:
// From commctrl.h
#ifdef UNICODE
#define HDN_BEGINTRACK HDN_BEGINTRACKW
#else
#define HDN_BEGINTRACK HDN_BEGINTRACKA
#endif
So when you implement a handler for HDN_BEGINTRACK, you're actually implementing it for HDN_BEGINTRACKA or HDN_BEGINTRACKW, depending on the value of UNICODE. But which message does the header control actually send? Remember, the header control is part of Windows, one of the common controls in comctl32.dll. Since the DLL is already compiled into executable code, changing the value of UNICODE in your project has absolutely no effect on its operation. How does the header control know which flavor of notification to send—A or W?
The answer lies in an oft-forgotten message, WM_NOTIFYFORMAT. When a control is first created, it sends a message to its parent, in effect asking, "do you want ANSI or Unicode notifications?" The parent responds with NFR_ANSI or NFR_UNICODE. If the parent doesn't handle WM_NOTIFYFORMAT, the Windows DefWindowProc responds based on the preference of the parent window or dialog itself. The default is Unicode. So I suspect the reason you had no luck trapping HDN_BEGINTRACK is that you compiled an ANSI program without handling WM_NOTIFYFORMAT. Your application is looking for HDN_BEGINTRACKA (ANSI) while the header control is sending HDN_BEGINTRACKW (Unicode).
One way to fix the problem is to implement a WM_NOTIFYFORMAT handler for your list control, one that returns NFR_ANSI. When I tried this, the header control did indeed send HDN_BEGINTRACKA and I was able to prevent sizing. But using NFR_ANSI broke other features. For example, the list control no longer repaints its columns while sizing.
A simpler, more reliable way to prevent sizing header columns is to implement handlers for both HDN_BEGINTRACKA and HDN_BEGINTRACKW. This not only obviates the need to process WM_NOTIFYFORMAT, it lets your code work in both ANSI and Unicode modes.
The header control sends HDN_XXX notifications to the parent (list control) window, but when using MFC you can use message reflection to handle the notifications in the header itself. Since the "lockable columns" feature is more a property of the header than the list control, this is the approach I chose. If you're not using MFC, you'll have to handle these notifications in the list control. To do message reflection, you can use ON_NOTIFY_REFLECT in your header control's message map or simply override the virtual function OnChildNotify, as shown here:
BOOL CLockableHeader::OnChildNotify(
UINT msg, WPARAM wp, LPARAM lp, LRESULT* pRes)
{
NMHDR& nmh = *(NMHDR*)lp;
if (nmh.code==HDN_BEGINTRACKW || nmg.code==HDN_BEGINTRACKA)
return *pRes=TRUE;
•••
}
Since OnChildNotify is virtual, there's no need for message map entries. All you have to do is implement it. In any given application, the header will send one or the other, not both. Either way, CLockableHeader eats the notification—that is, returns TRUE (handled) without passing on to the default header control. CLockableHeader controls locking through a flag m_bLocked which the app can set by calling CLockableHeader::Lock.
If you're going to prevent sizing, you should also disable the size cursor. Otherwise users may think your app is either broken or lame. Fortunately, it's trivial:
BOOL
CLockableHeader::OnSetCursor(
CWnd* pWnd, UINT nHit, UINT msg)
{
return m_bLocked ? TRUE :
CHeaderCtrl::OnSetCursor(pWnd, nHit, msg);
}
In other words: if the columns are locked, OnSetCursor returns TRUE without setting the cursor; otherwise, let the header control do its size cursor thing. Now when the columns are locked, Windows displays its standard arrow cursor instead of displaying the left-right sizing cursor.
Once you've implemented your custom header control by deriving from CHeaderCtrl, how do you get Windows to use it? The same way you would for any dialog control, by subclassing. The right place is in the parent window's OnCreate handler:
// CMyView is derived from CListView
int CMyView::OnCreate(LPCREATESTRUCT lpcs)
{
VERIFY(CListView::OnCreate(lpcs)==0);
return m_header.SubclassDlgItem(0,this) ? 0 : -1;
}
This works because the header control always has ID = 0. With all this in place, the only thing left to do is implement the command and UI update handlers for the View | Lock Columns command.
// From commctrl.h
#ifdef UNICODE
#define HDN_BEGINTRACK HDN_BEGINTRACKW
#else
#define HDN_BEGINTRACK HDN_BEGINTRACKA
#endif
So when you implement a handler for HDN_BEGINTRACK, you're actually implementing it for HDN_BEGINTRACKA or HDN_BEGINTRACKW, depending on the value of UNICODE. But which message does the header control actually send? Remember, the header control is part of Windows, one of the common controls in comctl32.dll. Since the DLL is already compiled into executable code, changing the value of UNICODE in your project has absolutely no effect on its operation. How does the header control know which flavor of notification to send—A or W?
The answer lies in an oft-forgotten message, WM_NOTIFYFORMAT. When a control is first created, it sends a message to its parent, in effect asking, "do you want ANSI or Unicode notifications?" The parent responds with NFR_ANSI or NFR_UNICODE. If the parent doesn't handle WM_NOTIFYFORMAT, the Windows DefWindowProc responds based on the preference of the parent window or dialog itself. The default is Unicode. So I suspect the reason you had no luck trapping HDN_BEGINTRACK is that you compiled an ANSI program without handling WM_NOTIFYFORMAT. Your application is looking for HDN_BEGINTRACKA (ANSI) while the header control is sending HDN_BEGINTRACKW (Unicode).
One way to fix the problem is to implement a WM_NOTIFYFORMAT handler for your list control, one that returns NFR_ANSI. When I tried this, the header control did indeed send HDN_BEGINTRACKA and I was able to prevent sizing. But using NFR_ANSI broke other features. For example, the list control no longer repaints its columns while sizing.
A simpler, more reliable way to prevent sizing header columns is to implement handlers for both HDN_BEGINTRACKA and HDN_BEGINTRACKW. This not only obviates the need to process WM_NOTIFYFORMAT, it lets your code work in both ANSI and Unicode modes.
The header control sends HDN_XXX notifications to the parent (list control) window, but when using MFC you can use message reflection to handle the notifications in the header itself. Since the "lockable columns" feature is more a property of the header than the list control, this is the approach I chose. If you're not using MFC, you'll have to handle these notifications in the list control. To do message reflection, you can use ON_NOTIFY_REFLECT in your header control's message map or simply override the virtual function OnChildNotify, as shown here:
BOOL CLockableHeader::OnChildNotify(
UINT msg, WPARAM wp, LPARAM lp, LRESULT* pRes)
{
NMHDR& nmh = *(NMHDR*)lp;
if (nmh.code==HDN_BEGINTRACKW || nmg.code==HDN_BEGINTRACKA)
return *pRes=TRUE;
•••
}
Since OnChildNotify is virtual, there's no need for message map entries. All you have to do is implement it. In any given application, the header will send one or the other, not both. Either way, CLockableHeader eats the notification—that is, returns TRUE (handled) without passing on to the default header control. CLockableHeader controls locking through a flag m_bLocked which the app can set by calling CLockableHeader::Lock.
If you're going to prevent sizing, you should also disable the size cursor. Otherwise users may think your app is either broken or lame. Fortunately, it's trivial:
BOOL
CLockableHeader::OnSetCursor(
CWnd* pWnd, UINT nHit, UINT msg)
{
return m_bLocked ? TRUE :
CHeaderCtrl::OnSetCursor(pWnd, nHit, msg);
}
In other words: if the columns are locked, OnSetCursor returns TRUE without setting the cursor; otherwise, let the header control do its size cursor thing. Now when the columns are locked, Windows displays its standard arrow cursor instead of displaying the left-right sizing cursor.
Once you've implemented your custom header control by deriving from CHeaderCtrl, how do you get Windows to use it? The same way you would for any dialog control, by subclassing. The right place is in the parent window's OnCreate handler:
// CMyView is derived from CListView
int CMyView::OnCreate(LPCREATESTRUCT lpcs)
{
VERIFY(CListView::OnCreate(lpcs)==0);
return m_header.SubclassDlgItem(0,this) ? 0 : -1;
}
This works because the header control always has ID = 0. With all this in place, the only thing left to do is implement the command and UI update handlers for the View | Lock Columns command.
Sunday, December 02, 2007
How lucky you are if you read this text
Docteur Phillip M Harter, Professeur à l'école de Médecine de l'Université de Stanford .
Si on pouvait réduire la population du monde en un village de 100 personnes tout en maintenant les proportions de tous les peuples existants sur la Terre, ce village serait ainsi composé :
57 asiatiques,
21 européens,
14 américains (Nord, Centre et Sud)
8 africains.
Il y aurait :
52 femmes et 48 hommes
30 blancs et 70 non blancs
30 chrétiens et 70 non chrétiens
89 hétérosexuels et 11 homosexuels
6 personnes posséderaient 59% de la richesse totale et tous les 6 seraient américains
80 vivraient dans des maisons vétustes
70 seraient analphabètes
50 souffriraient de malnutrition
1 serait en train de mourir
1 serait en train de naître
1 posséderait un ordinateur
1 (oui, un seulement) aurait un diplôme universitaire
Si on considère le monde de cette manière, le besoin d'accepter et de comprendre devient évident.
Prenez en considération aussi ceci :
Si vous vous êtes levé ce matin avec plus de santé que de maladie, vous êtes plus chanceux que le million de personnes qui ne verra pas la semaine prochaine.
Si vous n'avez jamais été dans le danger d'une bataille, la solitude de l'emprisonnement, l'agonie de la torture, l'étau de la faim, vous êtes mieux que 500 millions de personnes.
Si vous pouvez aller à l'église ou dans un temple sans peur d'être menacé, torturé ou tué, vous avez plus de chance que 3 milliards de personnes.
Si vous avez de la nourriture dans votre réfrigérateur, des habits sur vous, un toit sur votre tête et un endroit pour dormir, vous êtes plus riche que 75% des habitants de la Planète.
Si vous avez de l'argent à la banque, dans votre portefeuille ou de la monnaie dans une petite boite, vous faite partie des 8% les plus privilégiés du monde.
Si vos parents sont encore vivants et toujours mariés, vous êtes des personnes réellement rares.
Si, enfin, vous lisez ce message, vous venez de recevoir un double cadeau parce que quelqu'un a pensé à vous et parce que vous ne faites pas partie des deux milliards de personnes qui ne savent pas lire...
Si on pouvait réduire la population du monde en un village de 100 personnes tout en maintenant les proportions de tous les peuples existants sur la Terre, ce village serait ainsi composé :
57 asiatiques,
21 européens,
14 américains (Nord, Centre et Sud)
8 africains.
Il y aurait :
52 femmes et 48 hommes
30 blancs et 70 non blancs
30 chrétiens et 70 non chrétiens
89 hétérosexuels et 11 homosexuels
6 personnes posséderaient 59% de la richesse totale et tous les 6 seraient américains
80 vivraient dans des maisons vétustes
70 seraient analphabètes
50 souffriraient de malnutrition
1 serait en train de mourir
1 serait en train de naître
1 posséderait un ordinateur
1 (oui, un seulement) aurait un diplôme universitaire
Si on considère le monde de cette manière, le besoin d'accepter et de comprendre devient évident.
Prenez en considération aussi ceci :
Si vous vous êtes levé ce matin avec plus de santé que de maladie, vous êtes plus chanceux que le million de personnes qui ne verra pas la semaine prochaine.
Si vous n'avez jamais été dans le danger d'une bataille, la solitude de l'emprisonnement, l'agonie de la torture, l'étau de la faim, vous êtes mieux que 500 millions de personnes.
Si vous pouvez aller à l'église ou dans un temple sans peur d'être menacé, torturé ou tué, vous avez plus de chance que 3 milliards de personnes.
Si vous avez de la nourriture dans votre réfrigérateur, des habits sur vous, un toit sur votre tête et un endroit pour dormir, vous êtes plus riche que 75% des habitants de la Planète.
Si vous avez de l'argent à la banque, dans votre portefeuille ou de la monnaie dans une petite boite, vous faite partie des 8% les plus privilégiés du monde.
Si vos parents sont encore vivants et toujours mariés, vous êtes des personnes réellement rares.
Si, enfin, vous lisez ce message, vous venez de recevoir un double cadeau parce que quelqu'un a pensé à vous et parce que vous ne faites pas partie des deux milliards de personnes qui ne savent pas lire...
Monday, November 26, 2007
How to create a def file for a dll
you can still create a def file using the output from the objdump program (from the MinGW distribution). Here's an example.
objdump -p mydll.dll > mydll.fil
Search for "[Ordinal/Name Pointer] Table" in mydll.fil and use the list of functions following it to create your def file.
objdump -p mydll.dll > mydll.fil
Search for "[Ordinal/Name Pointer] Table" in mydll.fil and use the list of functions following it to create your def file.
Wednesday, November 21, 2007
Upload Video to Youtube
I am finally on the verge to integrate a youtube uploader tool to one of my side project. I just need to figure out how to log to gmail instead of youtube since there is two ways to authenticate with youtube.
About youtube :
When you upload a video to YouTube, it automatically converts it to a flash video file (.flv) with specific settings (see below, standard column). The contrast of the video is increased and video noise is suppressed. YouTube appears to respect the framerate of the uploaded file; Framerates will varie between 15 (minimum framerate to tell it is a video...), 24 frames (pal) and 30 frames per second (ntsc with no drop frame). It is hard to tell how long you have to wait for the video to be available but it is pretty fast !!. Once the video has been uploaded you can log to your account and from there you can check the status of your video. Usually you will see Uploaded (processing, please wait).
Video Profile
standard recommended possible
file extension : .flv (flash video)
total bitrate : ~320 kbits/sec 804.7 kbits/sec 2147.2 kbits/sec
audio codec : MPEG layer 3 (MP3)
audio format : Mono, 22.050 kHz Stereo, 44.100 kHz Stereo, 44.100 kHz
audio bitrate : ~67 kpbs/sec 128 kpbs/sec 128 kpbs/sec
video codec : Sorenson Spark (H.263)
video format : 320 x 240 px, 15-30 fps 320 x 240 px, 29.97 fps 640 x 480 px, 29.97 fps
video bitrate : ~250 kbps/sec 677.7 kbps/sec 2019.2 kbps/sec
You can save your own .flv files and upload them to YouTube. You can encode video with higher bitrates and resolution than YouTube compresses with. The only limitations I can tell are that the file must be under 100 MB and you must use the Sorenson Spark video codec and MP3 audio codec.
About youtube :
When you upload a video to YouTube, it automatically converts it to a flash video file (.flv) with specific settings (see below, standard column). The contrast of the video is increased and video noise is suppressed. YouTube appears to respect the framerate of the uploaded file; Framerates will varie between 15 (minimum framerate to tell it is a video...), 24 frames (pal) and 30 frames per second (ntsc with no drop frame). It is hard to tell how long you have to wait for the video to be available but it is pretty fast !!. Once the video has been uploaded you can log to your account and from there you can check the status of your video. Usually you will see Uploaded (processing, please wait).
Video Profile
standard recommended possible
file extension : .flv (flash video)
total bitrate : ~320 kbits/sec 804.7 kbits/sec 2147.2 kbits/sec
audio codec : MPEG layer 3 (MP3)
audio format : Mono, 22.050 kHz Stereo, 44.100 kHz Stereo, 44.100 kHz
audio bitrate : ~67 kpbs/sec 128 kpbs/sec 128 kpbs/sec
video codec : Sorenson Spark (H.263)
video format : 320 x 240 px, 15-30 fps 320 x 240 px, 29.97 fps 640 x 480 px, 29.97 fps
video bitrate : ~250 kbps/sec 677.7 kbps/sec 2019.2 kbps/sec
You can save your own .flv files and upload them to YouTube. You can encode video with higher bitrates and resolution than YouTube compresses with. The only limitations I can tell are that the file must be under 100 MB and you must use the Sorenson Spark video codec and MP3 audio codec.
Monday, November 19, 2007
How to take advantage of Silverlight Marker to create hypervideo
Interesting article from Jesse Liberty also i worked on Hypervideo back in 1999.... Silverlight is finally well suited to turn HD video into interactive sequences.
http://silverlight.net/blogs/jesseliberty/archive/2007/10/24/hypervideo-2.aspx
http://silverlight.net/blogs/jesseliberty/archive/2007/10/24/hypervideo-2.aspx
Wednesday, November 14, 2007
silent install of the VC 8.0 runtime (vcredist) packages
if you have downloaded the standalone VC 8.0 redistributable packages, you will need to modify the command lines slightly. The following command lines can be used to install the original release of the standalone VC 8.0 redistributable packages:
For x86: vcredist_x86.exe /q:a /c:"VCREDI~1.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For x64: vcredist_x64.exe /q:a /c:"VCREDI~2.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For ia64: vcredist_ia64.exe /q:a /c:"VCREDI~3.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
The following command lines can be used to install the Visual Studio 2005 SP1 release of the standalone VC 8.0 redistributable packages:
For x86: vcredist_x86.exe /q:a /c:"VCREDI~3.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For x64: vcredist_x64.exe /q:a /c:"VCREDI~2.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For ia64: vcredist_ia64.exe /q:a /c:"VCREDI~1.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
If you would like to install the VC runtime packages in unattended mode (which will show a small progress bar but not require any user interaction), you can change the /qn switch above to /qb. If you would like the progress bar to not show a cancel button, then you can change the /qn switch above to /qb!
Some folks ask me what vcredist.msi..
Iy means that you want to extract the embedded msi from the self exe package and be able to run msiexec which is used to install the package. msiexec is the Microsoft Installer runtime.
MSI is the prefered format for microsoft installer. Installshield and Visual Studio 2005 and higher can generate MSI. Using the msiexec command. You can even log all the installer session to a file and you can verify and explore the content of the MSI using ORCA in order to look at the installer sequence, string table and in some cases embedded Visual Basic Script. Yes Visual Basic is not dead. Still used and convnient enough for scripting installer code :)
If you want to install the runtime from a Null Soft Installer NSI script with minimal UI and progress bar, here is what you want to do
;extract selft exe to temp
SetOutPath "$TEMP"
; uncompress
File "${RUNTIME_VCREDIST}"
;-------------------------------
; Test if Visual Studio Redistributables 2005+ SP1 installed
; Returns -1 if there is no VC redistributables intstalled
Function CheckVCRedist
Push $R0
ClearErrors
ReadRegDword $R0 HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{7299052b-02a4-4627-81f2-1818da5d550d}" "Version"
; if VS 2005+ redist SP1 not installed, install it
IfErrors 0 VSRedistInstalled
StrCpy $R0 "-1"
VSRedistInstalled:
Exch $R0
FunctionEnd
Section "Installer Sections" SecDummy
SetOverwrite ifnewer
Call CheckVCRedist
;IntCmp $0 5 is5 lessthan5 morethan5
IntCmp $R0 -1 noVCRuntime noVCRuntime VCRuntime
;force install
; IntCmp $R0 -1 noVCRuntime noVCRuntime noVCRuntime
noVCRuntime:
;extract VCRuntime to Temp
SetOutPath "$TEMP"
File "${RUNTIME_VCREDIST}"
;/qn for complete silent install
;/qb small progress dialog but does not require any user interaction
;StrCpy $VC_RUNTIME "$\"$TEMP\vcredist_x86.exe$\" /q:a /c:$\"VCREDI~3.EXE /q:a /c:$\"$\"msiexec /i vcredist.msi /qb!$\""
StrCpy $VC_RUNTIME "$\"$TEMP\vcredist_x86.exe$\" /q:a"
;MessageBox MB_ICONINFORMATION 'Need to install VCRuntime {$R0} {$VC_RUNTIME}'
ExecWait '$VC_RUNTIME' $0
;MessageBox MB_ICONINFORMATION 'Return Code {$0}'
;IntCmp $0 0 done
;MessageBox MB_ICONINFORMATION 'install complete'
Delete "$TEMP\vcredist_x86.exe"
Goto VCRuntime
VCRuntime:
SectionEnd
http://blogs.msdn.com/astebner/archive/2007/02/07/update-regarding-silent-install-of-the-vc-8-0-runtime-vcredist-packages.aspx
For x86: vcredist_x86.exe /q:a /c:"VCREDI~1.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For x64: vcredist_x64.exe /q:a /c:"VCREDI~2.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For ia64: vcredist_ia64.exe /q:a /c:"VCREDI~3.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
The following command lines can be used to install the Visual Studio 2005 SP1 release of the standalone VC 8.0 redistributable packages:
For x86: vcredist_x86.exe /q:a /c:"VCREDI~3.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For x64: vcredist_x64.exe /q:a /c:"VCREDI~2.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
For ia64: vcredist_ia64.exe /q:a /c:"VCREDI~1.EXE /q:a /c:""msiexec /i vcredist.msi /qn"" "
If you would like to install the VC runtime packages in unattended mode (which will show a small progress bar but not require any user interaction), you can change the /qn switch above to /qb. If you would like the progress bar to not show a cancel button, then you can change the /qn switch above to /qb!
Some folks ask me what vcredist.msi..
Iy means that you want to extract the embedded msi from the self exe package and be able to run msiexec which is used to install the package. msiexec is the Microsoft Installer runtime.
MSI is the prefered format for microsoft installer. Installshield and Visual Studio 2005 and higher can generate MSI. Using the msiexec command. You can even log all the installer session to a file and you can verify and explore the content of the MSI using ORCA in order to look at the installer sequence, string table and in some cases embedded Visual Basic Script. Yes Visual Basic is not dead. Still used and convnient enough for scripting installer code :)
If you want to install the runtime from a Null Soft Installer NSI script with minimal UI and progress bar, here is what you want to do
;extract selft exe to temp
SetOutPath "$TEMP"
; uncompress
File "${RUNTIME_VCREDIST}"
;-------------------------------
; Test if Visual Studio Redistributables 2005+ SP1 installed
; Returns -1 if there is no VC redistributables intstalled
Function CheckVCRedist
Push $R0
ClearErrors
ReadRegDword $R0 HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{7299052b-02a4-4627-81f2-1818da5d550d}" "Version"
; if VS 2005+ redist SP1 not installed, install it
IfErrors 0 VSRedistInstalled
StrCpy $R0 "-1"
VSRedistInstalled:
Exch $R0
FunctionEnd
Section "Installer Sections" SecDummy
SetOverwrite ifnewer
Call CheckVCRedist
;IntCmp $0 5 is5 lessthan5 morethan5
IntCmp $R0 -1 noVCRuntime noVCRuntime VCRuntime
;force install
; IntCmp $R0 -1 noVCRuntime noVCRuntime noVCRuntime
noVCRuntime:
;extract VCRuntime to Temp
SetOutPath "$TEMP"
File "${RUNTIME_VCREDIST}"
;/qn for complete silent install
;/qb small progress dialog but does not require any user interaction
;StrCpy $VC_RUNTIME "$\"$TEMP\vcredist_x86.exe$\" /q:a /c:$\"VCREDI~3.EXE /q:a /c:$\"$\"msiexec /i vcredist.msi /qb!$\""
StrCpy $VC_RUNTIME "$\"$TEMP\vcredist_x86.exe$\" /q:a"
;MessageBox MB_ICONINFORMATION 'Need to install VCRuntime {$R0} {$VC_RUNTIME}'
ExecWait '$VC_RUNTIME' $0
;MessageBox MB_ICONINFORMATION 'Return Code {$0}'
;IntCmp $0 0 done
;MessageBox MB_ICONINFORMATION 'install complete'
Delete "$TEMP\vcredist_x86.exe"
Goto VCRuntime
VCRuntime:
SectionEnd
http://blogs.msdn.com/astebner/archive/2007/02/07/update-regarding-silent-install-of-the-vc-8-0-runtime-vcredist-packages.aspx
Sunday, November 11, 2007
Friday, October 12, 2007
Parsing XML in PHP
Traverse the document tree
In this example we have a XML file which consists of a document with a specific version. The document contains persons with the attributes firstname, lastname and description. The code snippet finds the document version and prints it. It's pretty straight forward you just check all the top nodes of the document and look for the one named document.
foreach ( $tree->children as $document )
{
// parse the document
if ( $document->name == "document" )
{
// get the document version attribute
foreach ( $document->attributes as $documentAttr )
{
if ( $documentAttr->name == "version" )
{
print( "Found document with version: " . $documentAttr->content . "
" );
}
}
// find persons here
}
}
When you've found the document node you can start looking for persons. This is done in the same manner, check the children nodes and look for person.
To make the process of getting the attribute values simpler we write a helper function to fetch the attribute value from a node.
function getAttrValue( $node, $attrName )
{
$ret = false;
foreach ( $node->attributes as $nodeAttr )
{
if ( $nodeAttr->name == $attrName )
{
$ret = $nodeAttr->content;
}
}
return $ret;
}
Now we're ready to parse the information describing the persons in this example. When we've found the person node we check all the subnodes and look for the nodes we want and fetch the information from these nodes.
// parse all persons
foreach ( $document->children as $person )
{
if ( $person->name == "person" )
{
print( "Found a new person
" );
$firstName = "";
$lastName = "";
$descriptionName = "";
// get the name and description
foreach ( $person->children as $personAttribute )
{
switch ( $personAttribute->name )
{
case "firstname" :
{
$firstName = getAttrValue( $personAttribute, "value" );
}break;
case "lastname" :
{
$lastName = getAttrValue( $personAttribute, "value" );
}break;
case "description" :
{
// get the description text
foreach ( $personAttribute->children as $description )
{
if ( $description->type == 3 )
{
$description = $description->content;
}
}
}break;
}
}
print( "The persons firstname is: $firstName
" );
print( "The persons lastname is: $lastName
" );
print( "The persons description is: $description
" );
}
}
include_once( "ezxml/classes/ezxml.php" );
$xmlDocument =
"
Coder.
Coder.
$tree =& eZXML::domTree( $xmlDocument, array( "TrimWhiteSpace" => true ) );
foreach ( $tree->children as $document )
{
// parse the document
if ( $document->name == "document" )
{
// get the document version attribute
foreach ( $document->attributes as $documentAttr )
{
if ( $documentAttr->name == "version" )
{
print( "Found document with version: " . $documentAttr->content . "
" );
}
}
// parse all persons
foreach ( $document->children as $person )
{
if ( $person->name == "person" )
{
print( "Found a new person
" );
$firstName = "";
$lastName = "";
$descriptionName = "";
// get the name and description
foreach ( $person->children as $personAttribute )
{
switch ( $personAttribute->name )
{
case "firstname" :
{
$firstName = getAttrValue( $personAttribute, "value" );
}break;
case "lastname" :
{
$lastName = getAttrValue( $personAttribute, "value" );
}break;
case "description" :
{
// get the description text
foreach ( $personAttribute->children as $description )
{
if ( $description->type == 3 )
{
$description = $description->content;
}
}
}break;
}
}
print( "The persons firstname is: $firstName
" );
print( "The persons lastname is: $lastName
" );
print( "The persons description is: $description
" );
}
}
}
}
/*!
Function to fetch an attribute value.
Will return the value of the attribute if found. False if not found.
*/
function getAttrValue( $node, $attrName )
{
$ret = false;
foreach ( $node->attributes as $nodeAttr )
{
if ( $nodeAttr->name == $attrName )
{
$ret = $nodeAttr->content;
}
}
return $ret;
}
This code will produce the following output:
Found document with version: 42
Found a new person
The persons firstname is: Bård
The persons lastname is: Farstad
The persons description is: Coder.
Found a new person
The persons firstname is: Christoffer A.
The persons lastname is: Elo
The persons description is: Coder2.
Using XML is simple and straightforward with the eZ xml class. You don't need any external libraries, the class produce the same document tree as you would get from the XML functions in PHP (which all need external libraries). It is also the only PHP XML parser class which returns the same object tree as the library functions, making it easy to use your programs both on sites where XML is compiled into PHP and sites where it isn't.
In this example we have a XML file which consists of a document with a specific version. The document contains persons with the attributes firstname, lastname and description. The code snippet finds the document version and prints it. It's pretty straight forward you just check all the top nodes of the document and look for the one named document.
foreach ( $tree->children as $document )
{
// parse the document
if ( $document->name == "document" )
{
// get the document version attribute
foreach ( $document->attributes as $documentAttr )
{
if ( $documentAttr->name == "version" )
{
print( "Found document with version: " . $documentAttr->content . "
" );
}
}
// find persons here
}
}
When you've found the document node you can start looking for persons. This is done in the same manner, check the children nodes and look for person.
To make the process of getting the attribute values simpler we write a helper function to fetch the attribute value from a node.
function getAttrValue( $node, $attrName )
{
$ret = false;
foreach ( $node->attributes as $nodeAttr )
{
if ( $nodeAttr->name == $attrName )
{
$ret = $nodeAttr->content;
}
}
return $ret;
}
Now we're ready to parse the information describing the persons in this example. When we've found the person node we check all the subnodes and look for the nodes we want and fetch the information from these nodes.
// parse all persons
foreach ( $document->children as $person )
{
if ( $person->name == "person" )
{
print( "Found a new person
" );
$firstName = "";
$lastName = "";
$descriptionName = "";
// get the name and description
foreach ( $person->children as $personAttribute )
{
switch ( $personAttribute->name )
{
case "firstname" :
{
$firstName = getAttrValue( $personAttribute, "value" );
}break;
case "lastname" :
{
$lastName = getAttrValue( $personAttribute, "value" );
}break;
case "description" :
{
// get the description text
foreach ( $personAttribute->children as $description )
{
if ( $description->type == 3 )
{
$description = $description->content;
}
}
}break;
}
}
print( "The persons firstname is: $firstName
" );
print( "The persons lastname is: $lastName
" );
print( "The persons description is: $description
" );
}
}
include_once( "ezxml/classes/ezxml.php" );
$xmlDocument =
"
Coder.
Coder.
$tree =& eZXML::domTree( $xmlDocument, array( "TrimWhiteSpace" => true ) );
foreach ( $tree->children as $document )
{
// parse the document
if ( $document->name == "document" )
{
// get the document version attribute
foreach ( $document->attributes as $documentAttr )
{
if ( $documentAttr->name == "version" )
{
print( "Found document with version: " . $documentAttr->content . "
" );
}
}
// parse all persons
foreach ( $document->children as $person )
{
if ( $person->name == "person" )
{
print( "Found a new person
" );
$firstName = "";
$lastName = "";
$descriptionName = "";
// get the name and description
foreach ( $person->children as $personAttribute )
{
switch ( $personAttribute->name )
{
case "firstname" :
{
$firstName = getAttrValue( $personAttribute, "value" );
}break;
case "lastname" :
{
$lastName = getAttrValue( $personAttribute, "value" );
}break;
case "description" :
{
// get the description text
foreach ( $personAttribute->children as $description )
{
if ( $description->type == 3 )
{
$description = $description->content;
}
}
}break;
}
}
print( "The persons firstname is: $firstName
" );
print( "The persons lastname is: $lastName
" );
print( "The persons description is: $description
" );
}
}
}
}
/*!
Function to fetch an attribute value.
Will return the value of the attribute if found. False if not found.
*/
function getAttrValue( $node, $attrName )
{
$ret = false;
foreach ( $node->attributes as $nodeAttr )
{
if ( $nodeAttr->name == $attrName )
{
$ret = $nodeAttr->content;
}
}
return $ret;
}
This code will produce the following output:
Found document with version: 42
Found a new person
The persons firstname is: Bård
The persons lastname is: Farstad
The persons description is: Coder.
Found a new person
The persons firstname is: Christoffer A.
The persons lastname is: Elo
The persons description is: Coder2.
Using XML is simple and straightforward with the eZ xml class. You don't need any external libraries, the class produce the same document tree as you would get from the XML functions in PHP (which all need external libraries). It is also the only PHP XML parser class which returns the same object tree as the library functions, making it easy to use your programs both on sites where XML is compiled into PHP and sites where it isn't.
Wednesday, September 26, 2007
PuTTY: A Free Telnet/SSH Client
I finally build putty for my Motorola Q VGA Phone Windows Mobile 5 Phone. The migration from eVC to visual studio 8 went well. I just need to fix some minor issue with some keyboard key not interpreted correctly when the host name need to be provided and figure out how to send specific keyboard combination key at once such as ctrl+c from a menu. Finally provide an installer to be able to have an icon in the tray. Without icons it is not possible for me to come back to putty once i need to answer a call.
Friday, August 31, 2007
Scene Cut Detection
Back in time i use to integrate on Windows the following software issued by
this famous French Laboratory. The purpose was to perform scene cut detection
thanks to a frame by frame video engine written for Windows and DirectShow.
The algorithm was very CPU intensive but pretty interesting.
http://www.irisa.fr/vista/Themes/Logiciel/MdShots/MdShots.english.html
this famous French Laboratory. The purpose was to perform scene cut detection
thanks to a frame by frame video engine written for Windows and DirectShow.
The algorithm was very CPU intensive but pretty interesting.
http://www.irisa.fr/vista/Themes/Logiciel/MdShots/MdShots.english.html
Monday, August 13, 2007
HTTP performance testing with httperf, autobench
HTTP performance testing with httperf, autobench
* httperf is a benchmarking tool that measures the HTTP request throughput of a web server. The way it achieves this is by sending requests to the server at a fixed rate and measuring the rate at which replies arrive. Running the test several times and with monotonically increasing request rates, one can see the reply rate level off when the server becomes saturated, i.e., when it is operating at its full capacity.
* autobench is a Perl wrapper around httperf. It runs httperf a number of times against a Web server, increasing the number of requested connections per second on each iteration, and extracts the significant data from the httperf output, delivering a CSV format file which can be imported directly into a spreadsheet for analysis/graphing.
I ran a series of autobench/httperf and openload tests against a Web site I'll call site2 in the following discussion (site2 is a beta version of a site I'll call site1). For comparison purposes, I also ran similar tests against site1 and against www.example.com. The machine I ran the tests from is a Red Hat 9 Linux server co-located in downtown Los Angeles.
Here is an example of running httperf against www.example.com:
# httperf --server=www.example.com --rate=10 --num-conns=500
httperf --client=0/1 --server=www.example.com --port=80 --uri=/ --rate=10 --send-buffer=4096 --recv-buffer=16384 --num-conns=500 --num-calls=1
Maximum connect burst length: 1
Total: connections 500 requests 500 replies 500 test-duration 50.354 s
Connection rate: 9.9 conn/s (100.7 ms/conn, <=8 concurrent connections)
Connection time [ms]: min 449.7 avg 465.1 max 2856.6 median 451.5 stddev 132.1
Connection time [ms]: connect 74.1
Connection length [replies/conn]: 1.000
Request rate: 9.9 req/s (100.7 ms/req)
Request size [B]: 65.0
Reply rate [replies/s]: min 9.2 avg 9.9 max 10.0 stddev 0.3 (10 samples)
Reply time [ms]: response 88.1 transfer 302.9
Reply size [B]: header 274.0 content 54744.0 footer 2.0 (total 55020.0)
Reply status: 1xx=0 2xx=500 3xx=0 4xx=0 5xx=0
CPU time [s]: user 15.65 system 34.65 (user 31.1% system 68.8% total 99.9%)
Net I/O: 534.1 KB/s (4.4*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
The 3 arguments I specified on the command line are:
* server: the name or IP address of your Web site (you can also specify a particular URL via the --uri argument)
* rate: specifies the number of HTTP requests/second sent to the Web server -- indicates the number of concurrent clients accessing the server
* num-conns: specifies how many total HTTP connections will be made during the test run -- this is a cumulative number, so the higher the number of connections, the longer the test run
Here is a detailed interpretation of an httperf test run. In short, the main numbers to look for are the connection rate, the request rate and the reply rate. Ideally, you would like to see that all these numbers are very close to the request rate specified on the command line. If the actual request rate and the reply rate start to decline, that's a sign your server became saturated and can't handle any new connections. That could also be a sign that your client became saturated, so that's why it's better to test your client against a fast Web site in order to gauge how many outgoing HTTP requests can be sustained by your client.
Autobench is a simple Perl script that facilitates multiple runs of httperf and automatically increases the HTTP request rate. Configuration of autobench can be achieved for example by means of the ~/.autobench.conf file. Here is how my file looks like:
# Autobench Configuration File
# host1, host2
# The hostnames of the servers under test
# Eg. host1 = iis.test.com
# host2 = apache.test.com
host1 = testhost1
host2 = testhost2
# uri1, uri2
# The URI to test (relative to the document root). For a fair comparison
# the files should be identical (although the paths to them may differ on the
# different hosts)
uri1 = /
uri2 = /
# port1, port2
# The port number on which the servers are listening
port1 = 80
port2 = 80
# low_rate, high_rate, rate_step
# The 'rate' is the number of number of connections to open per second.
# A series of tests will be conducted, starting at low rate,
# increasing by rate step, and finishing at high_rate.
# The default settings test at rates of 20,30,40,50...180,190,200
low_rate = 10
high_rate = 50
rate_step = 10
# num_conn, num_call
# num_conn is the total number of connections to make during a test
# num_call is the number of requests per connection
# The product of num_call and rate is the the approximate number of
# requests per second that will be attempted.
num_conn = 200
#num_call = 10
num_call = 1
# timeout sets the maximimum time (in seconds) that httperf will wait
# for replies from the web server. If the timeout is exceeded, the
# reply concerned is counted as an error.
timeout = 60
# output_fmt
# sets the output type - may be either "csv", or "tsv";
output_fmt = csv
## Config for distributed autobench (autobench_admin)
# clients
# comma separated list of the hostnames and portnumbers for the
# autobench clients. No whitespace can appear before or after the commas.
# clients = bench1.foo.com:4600,bench2.foo.com:4600,bench3.foo.com:4600
clients = localhost:4600
The only variable I usually tweak from one test run to another is num_conn, which I set to the desired number of total HTTP connections to the server for that test run. In the example file above it is set to 200.
I changed the default num_call value from 10 to 1 (num_call specifies the number of HTTP requests per connection; I like to set it to 1 to keep things simple). I started my test runs with low_rate set to 10, high_rate set to 50 and rate_step set to 10. What this means is that autobench will run httperf 5 times, starting with 10 requests/sec and going up to 50 requests/sec in increments of 10.
When running the following command line...
# autobench --single_host --host1=www.example.com --file=example.com.csv
* httperf is a benchmarking tool that measures the HTTP request throughput of a web server. The way it achieves this is by sending requests to the server at a fixed rate and measuring the rate at which replies arrive. Running the test several times and with monotonically increasing request rates, one can see the reply rate level off when the server becomes saturated, i.e., when it is operating at its full capacity.
* autobench is a Perl wrapper around httperf. It runs httperf a number of times against a Web server, increasing the number of requested connections per second on each iteration, and extracts the significant data from the httperf output, delivering a CSV format file which can be imported directly into a spreadsheet for analysis/graphing.
I ran a series of autobench/httperf and openload tests against a Web site I'll call site2 in the following discussion (site2 is a beta version of a site I'll call site1). For comparison purposes, I also ran similar tests against site1 and against www.example.com. The machine I ran the tests from is a Red Hat 9 Linux server co-located in downtown Los Angeles.
Here is an example of running httperf against www.example.com:
# httperf --server=www.example.com --rate=10 --num-conns=500
httperf --client=0/1 --server=www.example.com --port=80 --uri=/ --rate=10 --send-buffer=4096 --recv-buffer=16384 --num-conns=500 --num-calls=1
Maximum connect burst length: 1
Total: connections 500 requests 500 replies 500 test-duration 50.354 s
Connection rate: 9.9 conn/s (100.7 ms/conn, <=8 concurrent connections)
Connection time [ms]: min 449.7 avg 465.1 max 2856.6 median 451.5 stddev 132.1
Connection time [ms]: connect 74.1
Connection length [replies/conn]: 1.000
Request rate: 9.9 req/s (100.7 ms/req)
Request size [B]: 65.0
Reply rate [replies/s]: min 9.2 avg 9.9 max 10.0 stddev 0.3 (10 samples)
Reply time [ms]: response 88.1 transfer 302.9
Reply size [B]: header 274.0 content 54744.0 footer 2.0 (total 55020.0)
Reply status: 1xx=0 2xx=500 3xx=0 4xx=0 5xx=0
CPU time [s]: user 15.65 system 34.65 (user 31.1% system 68.8% total 99.9%)
Net I/O: 534.1 KB/s (4.4*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
The 3 arguments I specified on the command line are:
* server: the name or IP address of your Web site (you can also specify a particular URL via the --uri argument)
* rate: specifies the number of HTTP requests/second sent to the Web server -- indicates the number of concurrent clients accessing the server
* num-conns: specifies how many total HTTP connections will be made during the test run -- this is a cumulative number, so the higher the number of connections, the longer the test run
Here is a detailed interpretation of an httperf test run. In short, the main numbers to look for are the connection rate, the request rate and the reply rate. Ideally, you would like to see that all these numbers are very close to the request rate specified on the command line. If the actual request rate and the reply rate start to decline, that's a sign your server became saturated and can't handle any new connections. That could also be a sign that your client became saturated, so that's why it's better to test your client against a fast Web site in order to gauge how many outgoing HTTP requests can be sustained by your client.
Autobench is a simple Perl script that facilitates multiple runs of httperf and automatically increases the HTTP request rate. Configuration of autobench can be achieved for example by means of the ~/.autobench.conf file. Here is how my file looks like:
# Autobench Configuration File
# host1, host2
# The hostnames of the servers under test
# Eg. host1 = iis.test.com
# host2 = apache.test.com
host1 = testhost1
host2 = testhost2
# uri1, uri2
# The URI to test (relative to the document root). For a fair comparison
# the files should be identical (although the paths to them may differ on the
# different hosts)
uri1 = /
uri2 = /
# port1, port2
# The port number on which the servers are listening
port1 = 80
port2 = 80
# low_rate, high_rate, rate_step
# The 'rate' is the number of number of connections to open per second.
# A series of tests will be conducted, starting at low rate,
# increasing by rate step, and finishing at high_rate.
# The default settings test at rates of 20,30,40,50...180,190,200
low_rate = 10
high_rate = 50
rate_step = 10
# num_conn, num_call
# num_conn is the total number of connections to make during a test
# num_call is the number of requests per connection
# The product of num_call and rate is the the approximate number of
# requests per second that will be attempted.
num_conn = 200
#num_call = 10
num_call = 1
# timeout sets the maximimum time (in seconds) that httperf will wait
# for replies from the web server. If the timeout is exceeded, the
# reply concerned is counted as an error.
timeout = 60
# output_fmt
# sets the output type - may be either "csv", or "tsv";
output_fmt = csv
## Config for distributed autobench (autobench_admin)
# clients
# comma separated list of the hostnames and portnumbers for the
# autobench clients. No whitespace can appear before or after the commas.
# clients = bench1.foo.com:4600,bench2.foo.com:4600,bench3.foo.com:4600
clients = localhost:4600
The only variable I usually tweak from one test run to another is num_conn, which I set to the desired number of total HTTP connections to the server for that test run. In the example file above it is set to 200.
I changed the default num_call value from 10 to 1 (num_call specifies the number of HTTP requests per connection; I like to set it to 1 to keep things simple). I started my test runs with low_rate set to 10, high_rate set to 50 and rate_step set to 10. What this means is that autobench will run httperf 5 times, starting with 10 requests/sec and going up to 50 requests/sec in increments of 10.
When running the following command line...
# autobench --single_host --host1=www.example.com --file=example.com.csv
Wednesday, August 08, 2007
Source Code Beautifier for C, C++, C#, D, Java, and Pawn
A must to use !
http://uncrustify.sourceforge.net/
http://uncrustify.sourceforge.net/
how to convert IIS log to apache log using perl
# http://www.jammed.com/~jwa/hacks/iis2apache/iis2apache
# 1. Go to Start -> Control Panel -> Administrative Tools
# 2. Run Internet Information Services (IIS).
# 3. Find your Web site under the tree on the left.
# 4. Right-click on it and choose Properties.
# 5. On the Web site tab, you will see an option near the bottom that says "Active #Log Format." Click on the Properties button.
# 6. At the bottom of the General Properties tab, you will see a box that contains #the log file directory and the log file name. The full log path is comprised of the #log file directory plus the first part of the log file name.
#
#For example, if the dialog box displayed the following values:
#
# * Log file directory: C:\Windows\System32\LogFiles
# * Log file name: W3SVC1\exyymmdd.log
#Then your full log path would be:
#C:\Windows\System32\LogFiles\W3SVC1
#!/usr/bin/perl
#
# make an IIS log look strikingly like an apache log
# we make a vague attempt to interpret the Fields: header
# in an IIS logfile and use that to make IIS fields match up
# with apache fields.
#
# jwa@jammed.com 12 Dec 2000
#
while ($arg = shift @ARGV) {
$tzoffset = shift @ARGV if ($arg eq "--faketz");
$vhost = shift @ARGV if ($arg eq "--vhost");
$debug = 1 if ($arg eq "--debug"); # show field interpretation
}
if ($tzoffset eq "") {
print STDERR "Will use -0000 as a fake tzoffset\n";
$tzoffset = "-0000";
}
# build month hash
@m = ('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec');
while ($m = shift @m) {
$month{++$n} = $m;
}
# an IIS log adheres to what's defined by 'Fields;'
# attempt to parse this, tagging
LINE:
while ($line =) {
$line =~ s/\r|\n//g; # cooky DOS format
if ($line =~ /^#Fields: /) {
@line = split(" ", $line);
shift @line; # shifts of #Fields
# build a hash so we can look up a fieldname and
# have it return a position in the string
undef %fieldh;
$n = 0; # zero-based array for split
while ($l = shift @line) {
$fieldh{$l} = $n++;
print STDERR "$l is position $fieldh{$l}\n" if ($debug);
}
}
next LINE if ($line =~ /^#/);
#Fields: date time c-ip cs-username s-sitename s-computername s-ip cs-method cs-
#uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken
#s-port cs-version cs(User-Agent) cs(Cookie) cs(Referer)
# this is really slow.
$date = yankfield("date");
$time = yankfield("time");
$ip = yankfield("c-ip");
$username = yankfield("cs-username");
$method = yankfield("cs-method");
$stem = yankfield("cs-uri-stem");
$query = yankfield("cs-uri-query");
$status = yankfield("sc-status");
#$bytes = yankfield("cs-bytes");
# Which is it? sc-bytes or cs-bytes?
# cs-bytes only appears in some of the IIS logs I've seen.
# I'll assume that sc-bytes is "server->client bytes", which
# is what we want anyway.
$bytes = yankfield("sc-bytes"); # I'm gonna go with this.
$useragent = yankfield("cs(User-Agent)");
$referer = yankfield("cs(Referer)");
$useragent =~ s/\+/\ /g;
# our modified CLF sez:
# IP - - [DD/MMM/YYYY:HH:MM:SS TZOFFSET] "method stem[?query]" status bytes "referer" "user agent" "vhost"
# convert date
# 2000-07-19 00:00:01
($y, $m, $d) = split("-", $date);
$m =~ s/^0//g;
$mname = $month{$m};
# build url
$url = $stem;
if ($query ne "-") {
$url .= "?$query";
}
# all done, print it out
print "$ip - - [${d}/${mname}/${y}:${time} ${tzoffset}] \"$method $url\" $status $bytes \"$referer\" \"$useragent\" \"${vhost}\"\n";
}
# return the proper field, or "-" if it's not defined.
# (unfortunately ($date) = (split(" ", $line))[$fieldh{date}];
# will return element 0 if $fieldh{date} is undefined . . .)
sub yankfield {
my ($field) = shift @_;
print STDERR "Looking at $field; position [$fieldh{$field}]\n" if ($debug);
if ($fieldh{$field} ne "") {
return (split(" ", $line))[$fieldh{$field}];
} else {
print STDERR "$field undefined\n" if ($debug);
return "-";
}
}
# 1. Go to Start -> Control Panel -> Administrative Tools
# 2. Run Internet Information Services (IIS).
# 3. Find your Web site under the tree on the left.
# 4. Right-click on it and choose Properties.
# 5. On the Web site tab, you will see an option near the bottom that says "Active #Log Format." Click on the Properties button.
# 6. At the bottom of the General Properties tab, you will see a box that contains #the log file directory and the log file name. The full log path is comprised of the #log file directory plus the first part of the log file name.
#
#For example, if the dialog box displayed the following values:
#
# * Log file directory: C:\Windows\System32\LogFiles
# * Log file name: W3SVC1\exyymmdd.log
#Then your full log path would be:
#C:\Windows\System32\LogFiles\W3SVC1
#!/usr/bin/perl
#
# make an IIS log look strikingly like an apache log
# we make a vague attempt to interpret the Fields: header
# in an IIS logfile and use that to make IIS fields match up
# with apache fields.
#
# jwa@jammed.com 12 Dec 2000
#
while ($arg = shift @ARGV) {
$tzoffset = shift @ARGV if ($arg eq "--faketz");
$vhost = shift @ARGV if ($arg eq "--vhost");
$debug = 1 if ($arg eq "--debug"); # show field interpretation
}
if ($tzoffset eq "") {
print STDERR "Will use -0000 as a fake tzoffset\n";
$tzoffset = "-0000";
}
# build month hash
@m = ('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec');
while ($m = shift @m) {
$month{++$n} = $m;
}
# an IIS log adheres to what's defined by 'Fields;'
# attempt to parse this, tagging
LINE:
while ($line =
$line =~ s/\r|\n//g; # cooky DOS format
if ($line =~ /^#Fields: /) {
@line = split(" ", $line);
shift @line; # shifts of #Fields
# build a hash so we can look up a fieldname and
# have it return a position in the string
undef %fieldh;
$n = 0; # zero-based array for split
while ($l = shift @line) {
$fieldh{$l} = $n++;
print STDERR "$l is position $fieldh{$l}\n" if ($debug);
}
}
next LINE if ($line =~ /^#/);
#Fields: date time c-ip cs-username s-sitename s-computername s-ip cs-method cs-
#uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken
#s-port cs-version cs(User-Agent) cs(Cookie) cs(Referer)
# this is really slow.
$date = yankfield("date");
$time = yankfield("time");
$ip = yankfield("c-ip");
$username = yankfield("cs-username");
$method = yankfield("cs-method");
$stem = yankfield("cs-uri-stem");
$query = yankfield("cs-uri-query");
$status = yankfield("sc-status");
#$bytes = yankfield("cs-bytes");
# Which is it? sc-bytes or cs-bytes?
# cs-bytes only appears in some of the IIS logs I've seen.
# I'll assume that sc-bytes is "server->client bytes", which
# is what we want anyway.
$bytes = yankfield("sc-bytes"); # I'm gonna go with this.
$useragent = yankfield("cs(User-Agent)");
$referer = yankfield("cs(Referer)");
$useragent =~ s/\+/\ /g;
# our modified CLF sez:
# IP - - [DD/MMM/YYYY:HH:MM:SS TZOFFSET] "method stem[?query]" status bytes "referer" "user agent" "vhost"
# convert date
# 2000-07-19 00:00:01
($y, $m, $d) = split("-", $date);
$m =~ s/^0//g;
$mname = $month{$m};
# build url
$url = $stem;
if ($query ne "-") {
$url .= "?$query";
}
# all done, print it out
print "$ip - - [${d}/${mname}/${y}:${time} ${tzoffset}] \"$method $url\" $status $bytes \"$referer\" \"$useragent\" \"${vhost}\"\n";
}
# return the proper field, or "-" if it's not defined.
# (unfortunately ($date) = (split(" ", $line))[$fieldh{date}];
# will return element 0 if $fieldh{date} is undefined . . .)
sub yankfield {
my ($field) = shift @_;
print STDERR "Looking at $field; position [$fieldh{$field}]\n" if ($debug);
if ($fieldh{$field} ne "") {
return (split(" ", $line))[$fieldh{$field}];
} else {
print STDERR "$field undefined\n" if ($debug);
return "-";
}
}
Tuesday, July 17, 2007
Microsoft PhotoTours
This is an amazing piece of technology, presented at a TED conference.
http://www.collegehumor.com/video:1762315
More information is available on the Microsoft Research Website.
http://research.microsoft.com/IVM/PhotoTours/
There is also a live preview of this software available online. All you need to do is install a 5.5 MB Photosynth ActiveX. The address is :
http://labs.live.com/photosynth/
http://www.crazyleafdesign.com/blog/photosynth-prototype/
http://www.collegehumor.com/video:1762315
More information is available on the Microsoft Research Website.
http://research.microsoft.com/IVM/PhotoTours/
There is also a live preview of this software available online. All you need to do is install a 5.5 MB Photosynth ActiveX. The address is :
http://labs.live.com/photosynth/
http://www.crazyleafdesign.com/blog/photosynth-prototype/
Friday, July 13, 2007
Peter Löthberg mother's Sigbritt, 75, has world's fastest broadband
Sigbritt, 75, has world's fastest broadband
Published: 12th July 2007 11:07 CET
Online: http://www.thelocal.se/7869/
A 75 year old woman from Karlstad in central Sweden has been thrust into the IT history books - with the world's fastest internet connection.
Sigbritt Löthberg's home has been supplied with a blistering 40 Gigabits per second connection, many thousands of times faster than the average residential link and the first time ever that a home user has experienced such a high speed.
But Sigbritt, who had never had a computer until now, is no ordinary 75 year old. She is the mother of Swedish internet legend Peter Löthberg who, along with Karlstad Stadsnät, the local council's network arm, has arranged the connection.
"This is more than just a demonstration," said network boss Hafsteinn Jonsson.
"As a network owner we're trying to persuade internet operators to invest in faster connections. And Peter Löthberg wanted to show how you can build a low price, high capacity line over long distances," he told The Local.
Sigbritt will now be able to enjoy 1,500 high definition HDTV channels simultaneously. Or, if there is nothing worth watching there, she will be able to download a full high definition DVD in just two seconds.
The secret behind Sigbritt's ultra-fast connection is a new modulation technique which allows data to be transferred directly between two routers up to 2,000 kilometres apart, with no intermediary transponders.
According to Karlstad Stadsnät the distance is, in theory, unlimited - there is no data loss as long as the fibre is in place.
"I want to show that there are other methods than the old fashioned ways such as copper wires and radio, which lack the possibilities that fibre has," said Peter Löthberg, who now works at Cisco.
Cisco contributed to the project but the point, said Hafsteinn Jonsson, is that fibre technology makes such high speed connections technically and commercially viable.
"The most difficult part of the whole project was installing Windows on Sigbritt's PC," said Jonsson.
The Local (news@thelocal.se/08 656 6518)
Published: 12th July 2007 11:07 CET
Online: http://www.thelocal.se/7869/
A 75 year old woman from Karlstad in central Sweden has been thrust into the IT history books - with the world's fastest internet connection.
Sigbritt Löthberg's home has been supplied with a blistering 40 Gigabits per second connection, many thousands of times faster than the average residential link and the first time ever that a home user has experienced such a high speed.
But Sigbritt, who had never had a computer until now, is no ordinary 75 year old. She is the mother of Swedish internet legend Peter Löthberg who, along with Karlstad Stadsnät, the local council's network arm, has arranged the connection.
"This is more than just a demonstration," said network boss Hafsteinn Jonsson.
"As a network owner we're trying to persuade internet operators to invest in faster connections. And Peter Löthberg wanted to show how you can build a low price, high capacity line over long distances," he told The Local.
Sigbritt will now be able to enjoy 1,500 high definition HDTV channels simultaneously. Or, if there is nothing worth watching there, she will be able to download a full high definition DVD in just two seconds.
The secret behind Sigbritt's ultra-fast connection is a new modulation technique which allows data to be transferred directly between two routers up to 2,000 kilometres apart, with no intermediary transponders.
According to Karlstad Stadsnät the distance is, in theory, unlimited - there is no data loss as long as the fibre is in place.
"I want to show that there are other methods than the old fashioned ways such as copper wires and radio, which lack the possibilities that fibre has," said Peter Löthberg, who now works at Cisco.
Cisco contributed to the project but the point, said Hafsteinn Jonsson, is that fibre technology makes such high speed connections technically and commercially viable.
"The most difficult part of the whole project was installing Windows on Sigbritt's PC," said Jonsson.
The Local (news@thelocal.se/08 656 6518)
Monday, July 09, 2007
chown on linux using c++
lchown
NAME
lchown - change the owner and group of a symbolic link
#include
int lchown(const char *path, uid_t owner, gid_t group);
DESCRIPTION
The lchown() function shall be equivalent to chown(), except in the case where the named file is a symbolic link. In this case, lchown() shall change the ownership of the symbolic link file itself, while chown() changes the ownership of the file or directory to which the symbolic link refers.
RETURN VALUE
Upon successful completion, lchown() shall return 0. Otherwise, it shall return -1 and set errno to indicate an error.
ERRORS
The lchown() function shall fail if:
EACCES
Search permission is denied on a component of the path prefix of path.
EINVAL
The owner or group ID is not a value supported by the implementation.
ELOOP
A loop exists in symbolic links encountered during resolution of the path argument.
ENAMETOOLONG
The length of a pathname exceeds {PATH_MAX} or a pathname component is longer than {NAME_MAX}.
ENOENT
A component of path does not name an existing file or path is an empty string.
ENOTDIR
A component of the path prefix of path is not a directory.
EOPNOTSUPP
The path argument names a symbolic link and the implementation does not support setting the owner or group of a symbolic link.
EPERM
The effective user ID does not match the owner of the file and the process does not have appropriate privileges.
EROFS
The file resides on a read-only file system.
The lchown() function may fail if:
EIO
An I/O error occurred while reading or writing to the file system.
EINTR
A signal was caught during execution of the function.
ELOOP
More than {SYMLOOP_MAX} symbolic links were encountered during resolution of the path argument.
ENAMETOOLONG
Pathname resolution of a symbolic link produced an intermediate result whose length exceeds {PATH_MAX}.
The following sections are informative.
EXAMPLES
Changing the Current Owner of a File
The following example shows how to change the ownership of the symbolic link named /modules/pass1 to the user ID associated with "jones" and the group ID associated with "cnd".
The numeric value for the user ID is obtained by using the getpwnam() function. The numeric value for the group ID is obtained by using the getgrnam() function.
#include
#include
#include
#include
struct passwd *pwd;
struct group *grp;
char *path = "/modules/pass1";
...
pwd = getpwnam("Sri_Chinmoy");
grp = getgrnam("guru");
lchown(path, pwd->pw_uid, grp->gr_gid);
NAME
lchown - change the owner and group of a symbolic link
#include
int lchown(const char *path, uid_t owner, gid_t group);
DESCRIPTION
The lchown() function shall be equivalent to chown(), except in the case where the named file is a symbolic link. In this case, lchown() shall change the ownership of the symbolic link file itself, while chown() changes the ownership of the file or directory to which the symbolic link refers.
RETURN VALUE
Upon successful completion, lchown() shall return 0. Otherwise, it shall return -1 and set errno to indicate an error.
ERRORS
The lchown() function shall fail if:
EACCES
Search permission is denied on a component of the path prefix of path.
EINVAL
The owner or group ID is not a value supported by the implementation.
ELOOP
A loop exists in symbolic links encountered during resolution of the path argument.
ENAMETOOLONG
The length of a pathname exceeds {PATH_MAX} or a pathname component is longer than {NAME_MAX}.
ENOENT
A component of path does not name an existing file or path is an empty string.
ENOTDIR
A component of the path prefix of path is not a directory.
EOPNOTSUPP
The path argument names a symbolic link and the implementation does not support setting the owner or group of a symbolic link.
EPERM
The effective user ID does not match the owner of the file and the process does not have appropriate privileges.
EROFS
The file resides on a read-only file system.
The lchown() function may fail if:
EIO
An I/O error occurred while reading or writing to the file system.
EINTR
A signal was caught during execution of the function.
ELOOP
More than {SYMLOOP_MAX} symbolic links were encountered during resolution of the path argument.
ENAMETOOLONG
Pathname resolution of a symbolic link produced an intermediate result whose length exceeds {PATH_MAX}.
The following sections are informative.
EXAMPLES
Changing the Current Owner of a File
The following example shows how to change the ownership of the symbolic link named /modules/pass1 to the user ID associated with "jones" and the group ID associated with "cnd".
The numeric value for the user ID is obtained by using the getpwnam() function. The numeric value for the group ID is obtained by using the getgrnam() function.
#include
#include
#include
#include
struct passwd *pwd;
struct group *grp;
char *path = "/modules/pass1";
...
pwd = getpwnam("Sri_Chinmoy");
grp = getgrnam("guru");
lchown(path, pwd->pw_uid, grp->gr_gid);
Advanced Unix Programming Source
A set of handy function and class to perform some basics
UNIX tasks in c++
http://basepath.com/aup/ex/index.html
UNIX tasks in c++
http://basepath.com/aup/ex/index.html
Subscribe to:
Posts (Atom)