Sigbritt, 75, has world's fastest broadband
Published: 12th July 2007 11:07 CET
Online: http://www.thelocal.se/7869/
A 75 year old woman from Karlstad in central Sweden has been thrust into the IT history books - with the world's fastest internet connection.
Sigbritt Löthberg's home has been supplied with a blistering 40 Gigabits per second connection, many thousands of times faster than the average residential link and the first time ever that a home user has experienced such a high speed.
But Sigbritt, who had never had a computer until now, is no ordinary 75 year old. She is the mother of Swedish internet legend Peter Löthberg who, along with Karlstad Stadsnät, the local council's network arm, has arranged the connection.
"This is more than just a demonstration," said network boss Hafsteinn Jonsson.
"As a network owner we're trying to persuade internet operators to invest in faster connections. And Peter Löthberg wanted to show how you can build a low price, high capacity line over long distances," he told The Local.
Sigbritt will now be able to enjoy 1,500 high definition HDTV channels simultaneously. Or, if there is nothing worth watching there, she will be able to download a full high definition DVD in just two seconds.
The secret behind Sigbritt's ultra-fast connection is a new modulation technique which allows data to be transferred directly between two routers up to 2,000 kilometres apart, with no intermediary transponders.
According to Karlstad Stadsnät the distance is, in theory, unlimited - there is no data loss as long as the fibre is in place.
"I want to show that there are other methods than the old fashioned ways such as copper wires and radio, which lack the possibilities that fibre has," said Peter Löthberg, who now works at Cisco.
Cisco contributed to the project but the point, said Hafsteinn Jonsson, is that fibre technology makes such high speed connections technically and commercially viable.
"The most difficult part of the whole project was installing Windows on Sigbritt's PC," said Jonsson.
The Local (news@thelocal.se/08 656 6518)
Friday, July 13, 2007
Monday, July 09, 2007
chown on linux using c++
lchown
NAME
lchown - change the owner and group of a symbolic link
#include
int lchown(const char *path, uid_t owner, gid_t group);
DESCRIPTION
The lchown() function shall be equivalent to chown(), except in the case where the named file is a symbolic link. In this case, lchown() shall change the ownership of the symbolic link file itself, while chown() changes the ownership of the file or directory to which the symbolic link refers.
RETURN VALUE
Upon successful completion, lchown() shall return 0. Otherwise, it shall return -1 and set errno to indicate an error.
ERRORS
The lchown() function shall fail if:
EACCES
Search permission is denied on a component of the path prefix of path.
EINVAL
The owner or group ID is not a value supported by the implementation.
ELOOP
A loop exists in symbolic links encountered during resolution of the path argument.
ENAMETOOLONG
The length of a pathname exceeds {PATH_MAX} or a pathname component is longer than {NAME_MAX}.
ENOENT
A component of path does not name an existing file or path is an empty string.
ENOTDIR
A component of the path prefix of path is not a directory.
EOPNOTSUPP
The path argument names a symbolic link and the implementation does not support setting the owner or group of a symbolic link.
EPERM
The effective user ID does not match the owner of the file and the process does not have appropriate privileges.
EROFS
The file resides on a read-only file system.
The lchown() function may fail if:
EIO
An I/O error occurred while reading or writing to the file system.
EINTR
A signal was caught during execution of the function.
ELOOP
More than {SYMLOOP_MAX} symbolic links were encountered during resolution of the path argument.
ENAMETOOLONG
Pathname resolution of a symbolic link produced an intermediate result whose length exceeds {PATH_MAX}.
The following sections are informative.
EXAMPLES
Changing the Current Owner of a File
The following example shows how to change the ownership of the symbolic link named /modules/pass1 to the user ID associated with "jones" and the group ID associated with "cnd".
The numeric value for the user ID is obtained by using the getpwnam() function. The numeric value for the group ID is obtained by using the getgrnam() function.
#include
#include
#include
#include
struct passwd *pwd;
struct group *grp;
char *path = "/modules/pass1";
...
pwd = getpwnam("Sri_Chinmoy");
grp = getgrnam("guru");
lchown(path, pwd->pw_uid, grp->gr_gid);
NAME
lchown - change the owner and group of a symbolic link
#include
int lchown(const char *path, uid_t owner, gid_t group);
DESCRIPTION
The lchown() function shall be equivalent to chown(), except in the case where the named file is a symbolic link. In this case, lchown() shall change the ownership of the symbolic link file itself, while chown() changes the ownership of the file or directory to which the symbolic link refers.
RETURN VALUE
Upon successful completion, lchown() shall return 0. Otherwise, it shall return -1 and set errno to indicate an error.
ERRORS
The lchown() function shall fail if:
EACCES
Search permission is denied on a component of the path prefix of path.
EINVAL
The owner or group ID is not a value supported by the implementation.
ELOOP
A loop exists in symbolic links encountered during resolution of the path argument.
ENAMETOOLONG
The length of a pathname exceeds {PATH_MAX} or a pathname component is longer than {NAME_MAX}.
ENOENT
A component of path does not name an existing file or path is an empty string.
ENOTDIR
A component of the path prefix of path is not a directory.
EOPNOTSUPP
The path argument names a symbolic link and the implementation does not support setting the owner or group of a symbolic link.
EPERM
The effective user ID does not match the owner of the file and the process does not have appropriate privileges.
EROFS
The file resides on a read-only file system.
The lchown() function may fail if:
EIO
An I/O error occurred while reading or writing to the file system.
EINTR
A signal was caught during execution of the function.
ELOOP
More than {SYMLOOP_MAX} symbolic links were encountered during resolution of the path argument.
ENAMETOOLONG
Pathname resolution of a symbolic link produced an intermediate result whose length exceeds {PATH_MAX}.
The following sections are informative.
EXAMPLES
Changing the Current Owner of a File
The following example shows how to change the ownership of the symbolic link named /modules/pass1 to the user ID associated with "jones" and the group ID associated with "cnd".
The numeric value for the user ID is obtained by using the getpwnam() function. The numeric value for the group ID is obtained by using the getgrnam() function.
#include
#include
#include
#include
struct passwd *pwd;
struct group *grp;
char *path = "/modules/pass1";
...
pwd = getpwnam("Sri_Chinmoy");
grp = getgrnam("guru");
lchown(path, pwd->pw_uid, grp->gr_gid);
Advanced Unix Programming Source
A set of handy function and class to perform some basics
UNIX tasks in c++
http://basepath.com/aup/ex/index.html
UNIX tasks in c++
http://basepath.com/aup/ex/index.html
Thursday, July 05, 2007
Thread POSIX Thread basic
Further Threads Programming:Thread Attributes (POSIX)
PART I
API covered is for POSIX threads only. Otherwise, the functionality for Solaris threads and pthreads is largely the same.
Attributes
Attributes are a way to specify behavior that is different from the default. When a thread is created with pthread_create() or when a synchronization variable is initialized, an attribute object can be specified. Note: however that the default atributes are usually sufficient for most applications.
Important Note: Attributes are specified only at thread creation time; they cannot be altered while the thread is being used.
Thus three functions are usually called in tandem
* Thread attibute intialisation -- pthread_attr_init() create a default pthread_attr_t tattr
* Thread attribute value change (unless defaults appropriate) -- a variety of pthread_attr_*() functions are available to set individual attribute values for the pthread_attr_t tattr structure. (see below).
* Thread creation -- a call to pthread_create() with approriate attribute values set in a pthread_attr_t tattr structure.
The following code fragment should make this point clearer:
#include
pthread_attr_t tattr;
pthread_t tid;
void *start_routine;
void arg
int ret;
/* initialized with default attributes */
ret = pthread_attr_init(&tattr);
/* call an appropriate functions to alter a default value */
ret = pthread_attr_*(&tattr,SOME_ATRIBUTE_VALUE_PARAMETER);
/* create the thread */
ret = pthread_create(&tid, &tattr, start_routine, arg);
In order to save space, code examples mainly focus on the attribute setting functions and the intializing and creation functions are ommitted. These must of course be present in all actual code fragtments.
An attribute object is opaque, and cannot be directly modified by assignments. A set of functions is provided to initialize, configure, and destroy each object type. Once an attribute is initialized and configured, it has process-wide scope. The suggested method for using attributes is to configure all required state specifications at one time in the early stages of program execution. The appropriate attribute object can then be referred to as needed. Using attribute objects has two primary advantages:
* First, it adds to code portability. Even though supported attributes might vary between implementations, you need not modify function calls that create thread entities because the attribute object is hidden from the interface. If the target port supports attributes that are not found in the current port, provision must be made to manage the new attributes. This is an easy porting task though, because attribute objects need only be initialized once in a well-defined location.
* Second, state specification in an application is simplified. As an example, consider that several sets of threads might exist within a process, each providing a separate service, and each with its own state requirements. At some point in the early stages of the application, a thread attribute object can be initialized for each set. All future thread creations will then refer to the attribute object initialized for that type of thread. The initialization phase is simple and localized, and any future modifications can be made quickly and reliably.
Attribute objects require attention at process exit time. When the object is initialized, memory is allocated for it. This memory must be returned to the system. The pthreads standard provides function calls to destroy attribute objects.
Initializing Thread Attributes
The function pthread_attr_init() is used to initialize object attributes to their default values. The storage is allocated by the thread system during execution.
The function is prototyped by:
int pthread_attr_init(pthread_attr_t *tattr);
An example call to this function is:
#include
pthread_attr_t tattr;
int ret;
/* initialize an attribute to the default value */
ret = pthread_attr_init(&tattr);
The default values for attributes (tattr) are:
Attribute Value Result
scope PTHREAD_SCOPE_PROCESS New thread is unbound - not permanently attached to LWP.
detachstate PTHREAD_CREATE_JOINABLE Exit status and thread are preserved after the thread
terminates.
stackaddr NULL New thread has system-allocated stack address.
stacksize 1 megabyte New thread has system-defined stack size.
priority New thread inherits parent thread priority.
inheritsched PTHREAD_INHERIT_SCHED New thread inherits parent thread scheduling priority.
schedpolicy SCHED_OTHER New thread uses Solaris-defined fixed priority scheduling;
threads run until preempted by a higher-priority thread or until they block or yield.
This function zero after completing successfully. Any other returned value indicates that an error occurred. If the following condition occurs, the function fails and returns an error value (to errno).
Destroying Thread Attributes
The function pthread_attr_destroy() is used to remove the storage allocated during initialization. The attribute object becomes invalid. It is prototyped by:
int pthread_attr_destroy(pthread_attr_t *tattr);
A sample call to this functions is:
#include
pthread_attr_t tattr;
int ret;
/* destroy an attribute */
ret = pthread_attr_destroy(&tattr);
Attribites are declared as for pthread_attr_init() above.
pthread_attr_destroy() returns zero after completing successfully. Any other returned value indicates that an error occurred.
Thread's Detach State
When a thread is created detached (PTHREAD_CREATE_DETACHED), its thread ID and other resources can be reused as soon as the thread terminates.
If you do not want the calling thread to wait for the thread to terminate then call the function pthread_attr_setdetachstate().
When a thread is created nondetached (PTHREAD_CREATE_JOINABLE), it is assumed that you will be waiting for it. That is, it is assumed that you will be executing a pthread_join() on the thread. Whether a thread is created detached or nondetached, the process does not exit until all threads have exited.
pthread_attr_setdetachstate() is prototyped by:
int pthread_attr_setdetachstate(pthread_attr_t *tattr,int detachstate);
pthread_attr_setdetachstate() returns zero after completing successfully. Any other returned value indicates that an error occurred. If the following condition occurs, the function fails and returns the corresponding value.
An example call to detatch a thread with this function is:
#include
pthread_attr_t tattr;
int ret;
/* set the thread detach state */
ret = pthread_attr_setdetachstate(&tattr,PTHREAD_CREATE_DETACHED);
Note - When there is no explicit synchronization to prevent it, a newly created, detached thread can die and have its thread ID reassigned to another new thread before its creator returns from pthread_create(). For nondetached (PTHREAD_CREATE_JOINABLE) threads, it is very important that some thread join with it after it terminates -- otherwise the resources of that thread are not released for use by new threads. This commonly results in a memory leak. So when you do not want a thread to be joined, create it as a detached thread.
It is quite common that you will wish to create a thread which is detatched from creation. The following code illustrates how this may be achieved with the standard calls to initialise and set and then create a thread:
#include
pthread_attr_t tattr;
pthread_t tid;
void *start_routine;
void arg
int ret;
/* initialized with default attributes */
ret = pthread_attr_init(&tattr);
ret = pthread_attr_setdetachstate(&tattr,PTHREAD_CREATE_DETACHED);
ret = pthread_create(&tid, &tattr, start_routine, arg);
The function pthread_attr_getdetachstate() may be used to retrieve the thread create state, which can be either detached or joined. It is prototyped by:
int pthread_attr_getdetachstate(const pthread_attr_t *tattr, int *detachstate);
pthread_attr_getdetachstate() returns zero after completing successfully. Any other returned value indicates that an error occurred.
An example call to this fuction is:
#include
pthread_attr_t tattr;
int detachstate;
int ret;
/* get detachstate of thread */
ret = pthread_attr_getdetachstate (&tattr, &detachstate);
Full Article here
http://www.cs.cf.ac.uk/Dave/C/node30.html
PART II
Meanwhile, the second thread will do something like this:
Assuming both threads use the same mutex, we are assured that after they both ran through this code, either both variables are set to '0', or both are set to '1'. You'd note this requires some work from the programmer - If a third thread was to access these variables via some code that does not use this mutex, it still might mess up the variable's contents. Thus, it is important to enclose all the code that accesses these variables in a small set of functions, and always use only these functions to access these variables.
One note should be made here: This type of initialization creates a mutex called 'fast mutex'. This means that if a thread locks the mutex and then tries to lock it again, it'll get stuck - it will be in a deadlock.
There is another type of mutex, called 'recursive mutex', which allows the thread that locked it, to lock it several more times, without getting blocked (but other threads that try to lock the mutex now will get blocked). If the thread then unlocks the mutex, it'll still be locked, until it is unlocked the same amount of times as it was locked. This is similar to the way modern door locks work - if you turned it twice clockwise to lock it, you need to turn it twice counter-clockwise to unlock it. This kind of mutex can be created by assigning the constant
After the thread did what it had to (change variables or data structures, handle file, or whatever it intended to do), it should free the mutex, using the
After this call, this variable (a_mutex) may not be used as a mutex any more, unless it is initialized again. Thus, if one destroys a mutex too early, and another thread tries to lock or unlock it, that thread will get a
Two copies of the program are supplied. One that uses a mutex, and one that does not. Try them both, to see the differences, and be convinced that mutexes are essential in a multi-threaded environment.
The programs themselves are in the files accompanying this tutorial. The one that uses a mutex is employee-with-mutex.c. The one that does not use a mutex is employee-without-mutex.c. Read the comments inside the source files to get a better understanding of how they work.
Again we should remember that
The pthread library might, however, figure out a "deadlock". A deadlock is a situation in which a set of threads are all waiting for resources taken by other threads, all in the same set. Naturally, if all threads are blocked waiting for a mutex, none of them will ever come back to life again. The pthread library keeps track of such situations, and thus would fail the last thread trying to call
All these examples require the ability to send notifications between threads. This is where condition variables are brought into the picture.
Note that a condition variable does not provide locking. Thus, a mutex is used along with the condition variable, to provide the necessary locking when accessing this condition variable.
This defines a condition variable named 'got_request', and initializes it.
Note: since the
Or by using the broadcast function:
When either function returns, 'rc' is set to 0 on success, and to a non-zero value on failure. In such a case (failure), the return value denotes the error that occured (
Note: success of a signaling operation does not mean any thread was awakened - it might be that no thread was waiting on the condition variable, and thus the signaling does nothing (i.e. the signal is lost).
It is also not remembered for future use - if after the signaling function returns another thread starts waiting on this condition variable, a further signal is required to wake it up.
The only difference between these two functions is that
Here is how to use these two functions. We make the assumption that 'got_request' is a properly initialized condition variable, and that 'request_mutex' is a properly initialized mutex. First, we try the
Now an example using the
As you can see, the timed wait version is way more complex, and thus better be wrapped up by some function, rather then being re-coded in every necessary location.
Note: it might be that a condition variable that has 2 or more threads waiting on it is signaled many times, and yet one of the threads waiting on it never awakened. This is because we are not guaranteed which of the waiting threads is awakened when the variable is signaled. It might be that the awakened thread quickly comes back to waiting on the condition variables, and gets awakened again when the variable is signaled again, and so on. The situation for the un-awakened thread is called 'starvation'. It is up to the programmer to make sure this situation does not occur if it implies bad behavior. Yet, in our server example from before, this situation might indicate requests are coming in a very slow pace, and thus perhaps we have too many threads waiting to service requests. In this case, this situation is actually good, as it means every request is handled immediately when it arrives.
Note 2: when the mutex is being broadcast (using pthread_cond_broadcast), this does not mean all threads are running together. Each of them tries to lock the mutex again before returning from their wait function, and thus they'll start running one by one, each one locking the mutex, doing their work, and freeing the mutex before the next thread gets its chance to run.
What if some thread is still waiting on this variable? depending on the case, it might imply some flaw in the usage of this variable, or just lack of proper thread cleanup code. It is probably good to alert the programmer, at least during debug phase of the program, of such a case. It might mean nothing, but it might be significant.
However, what if all threads are busy handling previous requests, when a new one arrives? the signaling of the condition variable will do nothing (since all threads are busy doing other things, NOT waiting on the condition variable now), and after all threads finish handling their current request, they come back to wait on the variable, which won't necessarily be signaled again (for example, if no new requests arrive). Thus, there is at least one request pending, while all handling threads are blocked, waiting for a signal.
In order to overcome this problem, we may set some integer variable to denote the number of pending requests, and have each thread check the value of this variable before waiting on the variable. If this variable's value is positive, some request is pending, and the thread should go and handle it, instead of going to sleep. Further more, a thread that handled a request, should reduce the value of this variable by one, to make the count correct.
Lets see how this affects the waiting code we have seen above.
PART I
API covered is for POSIX threads only. Otherwise, the functionality for Solaris threads and pthreads is largely the same.
Attributes
Attributes are a way to specify behavior that is different from the default. When a thread is created with pthread_create() or when a synchronization variable is initialized, an attribute object can be specified. Note: however that the default atributes are usually sufficient for most applications.
Important Note: Attributes are specified only at thread creation time; they cannot be altered while the thread is being used.
Thus three functions are usually called in tandem
* Thread attibute intialisation -- pthread_attr_init() create a default pthread_attr_t tattr
* Thread attribute value change (unless defaults appropriate) -- a variety of pthread_attr_*() functions are available to set individual attribute values for the pthread_attr_t tattr structure. (see below).
* Thread creation -- a call to pthread_create() with approriate attribute values set in a pthread_attr_t tattr structure.
The following code fragment should make this point clearer:
#include
pthread_attr_t tattr;
pthread_t tid;
void *start_routine;
void arg
int ret;
/* initialized with default attributes */
ret = pthread_attr_init(&tattr);
/* call an appropriate functions to alter a default value */
ret = pthread_attr_*(&tattr,SOME_ATRIBUTE_VALUE_PARAMETER);
/* create the thread */
ret = pthread_create(&tid, &tattr, start_routine, arg);
In order to save space, code examples mainly focus on the attribute setting functions and the intializing and creation functions are ommitted. These must of course be present in all actual code fragtments.
An attribute object is opaque, and cannot be directly modified by assignments. A set of functions is provided to initialize, configure, and destroy each object type. Once an attribute is initialized and configured, it has process-wide scope. The suggested method for using attributes is to configure all required state specifications at one time in the early stages of program execution. The appropriate attribute object can then be referred to as needed. Using attribute objects has two primary advantages:
* First, it adds to code portability. Even though supported attributes might vary between implementations, you need not modify function calls that create thread entities because the attribute object is hidden from the interface. If the target port supports attributes that are not found in the current port, provision must be made to manage the new attributes. This is an easy porting task though, because attribute objects need only be initialized once in a well-defined location.
* Second, state specification in an application is simplified. As an example, consider that several sets of threads might exist within a process, each providing a separate service, and each with its own state requirements. At some point in the early stages of the application, a thread attribute object can be initialized for each set. All future thread creations will then refer to the attribute object initialized for that type of thread. The initialization phase is simple and localized, and any future modifications can be made quickly and reliably.
Attribute objects require attention at process exit time. When the object is initialized, memory is allocated for it. This memory must be returned to the system. The pthreads standard provides function calls to destroy attribute objects.
Initializing Thread Attributes
The function pthread_attr_init() is used to initialize object attributes to their default values. The storage is allocated by the thread system during execution.
The function is prototyped by:
int pthread_attr_init(pthread_attr_t *tattr);
An example call to this function is:
#include
pthread_attr_t tattr;
int ret;
/* initialize an attribute to the default value */
ret = pthread_attr_init(&tattr);
The default values for attributes (tattr) are:
Attribute Value Result
scope PTHREAD_SCOPE_PROCESS New thread is unbound - not permanently attached to LWP.
detachstate PTHREAD_CREATE_JOINABLE Exit status and thread are preserved after the thread
terminates.
stackaddr NULL New thread has system-allocated stack address.
priority New thread inherits parent thread priority.
inheritsched PTHREAD_INHERIT_SCHED New thread inherits parent thread scheduling priority.
schedpolicy SCHED_OTHER New thread uses Solaris-defined fixed priority scheduling;
threads run until preempted by a higher-priority thread or until they block or yield.
This function zero after completing successfully. Any other returned value indicates that an error occurred. If the following condition occurs, the function fails and returns an error value (to errno).
Destroying Thread Attributes
The function pthread_attr_destroy() is used to remove the storage allocated during initialization. The attribute object becomes invalid. It is prototyped by:
int pthread_attr_destroy(pthread_attr_t *tattr);
A sample call to this functions is:
#include
pthread_attr_t tattr;
int ret;
/* destroy an attribute */
ret = pthread_attr_destroy(&tattr);
Attribites are declared as for pthread_attr_init() above.
pthread_attr_destroy() returns zero after completing successfully. Any other returned value indicates that an error occurred.
Thread's Detach State
When a thread is created detached (PTHREAD_CREATE_DETACHED), its thread ID and other resources can be reused as soon as the thread terminates.
If you do not want the calling thread to wait for the thread to terminate then call the function pthread_attr_setdetachstate().
When a thread is created nondetached (PTHREAD_CREATE_JOINABLE), it is assumed that you will be waiting for it. That is, it is assumed that you will be executing a pthread_join() on the thread. Whether a thread is created detached or nondetached, the process does not exit until all threads have exited.
pthread_attr_setdetachstate() is prototyped by:
int pthread_attr_setdetachstate(pthread_attr_t *tattr,int detachstate);
pthread_attr_setdetachstate() returns zero after completing successfully. Any other returned value indicates that an error occurred. If the following condition occurs, the function fails and returns the corresponding value.
An example call to detatch a thread with this function is:
#include
pthread_attr_t tattr;
int ret;
/* set the thread detach state */
ret = pthread_attr_setdetachstate(&tattr,PTHREAD_CREATE_DETACHED);
Note - When there is no explicit synchronization to prevent it, a newly created, detached thread can die and have its thread ID reassigned to another new thread before its creator returns from pthread_create(). For nondetached (PTHREAD_CREATE_JOINABLE) threads, it is very important that some thread join with it after it terminates -- otherwise the resources of that thread are not released for use by new threads. This commonly results in a memory leak. So when you do not want a thread to be joined, create it as a detached thread.
It is quite common that you will wish to create a thread which is detatched from creation. The following code illustrates how this may be achieved with the standard calls to initialise and set and then create a thread:
#include
pthread_attr_t tattr;
pthread_t tid;
void *start_routine;
void arg
int ret;
/* initialized with default attributes */
ret = pthread_attr_init(&tattr);
ret = pthread_attr_setdetachstate(&tattr,PTHREAD_CREATE_DETACHED);
ret = pthread_create(&tid, &tattr, start_routine, arg);
The function pthread_attr_getdetachstate() may be used to retrieve the thread create state, which can be either detached or joined. It is prototyped by:
int pthread_attr_getdetachstate(const pthread_attr_t *tattr, int *detachstate);
pthread_attr_getdetachstate() returns zero after completing successfully. Any other returned value indicates that an error occurred.
An example call to this fuction is:
#include
pthread_attr_t tattr;
int detachstate;
int ret;
/* get detachstate of thread */
ret = pthread_attr_getdetachstate (&tattr, &detachstate);
Full Article here
http://www.cs.cf.ac.uk/Dave/C/node30.html
What Is a Thread? Why Use Threads
A thread is a semi-process, that has its own stack, and executes a given piece of code. Unlike a real process, the thread normally shares its memory with other threads (where as for processes we usually have a different memory area for each one of them). A Thread Group is a set of threads all executing inside the same process. They all share the same memory, and thus can access the same global variables, same heap memory, same set of file descriptors, etc. All these threads execute in parallel (i.e. using time slices, or if the system has several processors, then really in parallel).
The advantage of using a thread group instead of a normal serial program is that several operations may be carried out in parallel, and thus events can be handled immediately as they arrive (for example, if we have one thread handling a user interface, and another thread handling database queries, we can execute a heavy query requested by the user, and still respond to user input while the query is executed).
The advantage of using a thread group over using a process group is that context switching between threads is much faster then context switching between processes (context switching means that the system switches from running one thread or process, to running another thread or process). Also, communications between two threads is usually faster and easier to implement then communications between two processes.
On the other hand, because threads in a group all use the same memory space, if one of them corrupts the contents of its memory, other threads might suffer as well. With processes, the operating system normally protects processes from one another, and thus if one corrupts its own memory space, other processes won't suffer. Another advantage of using processes is that they can run on different machines, while all the threads have to run on the same machine (at least normally).
Creating And Destroying Threads
When a multi-threaded program starts executing, it has one thread running, which executes the main() function of the program. This is already a full-fledged thread, with its own thread ID. In order to create a new thread, the program should use the
pthread_create() function. Here is how to use it:
#include /* standard I/O routines */
#include /* pthread functions and data structures */
/* function to be executed by the new thread */
void*
do_loop(void* data)
{
int i;
int i; /* counter, to print numbers */
int j; /* counter, for delay */
int me = *((int*)data); /* thread identifying number */
for (i=0; i<10 color="brown" font="" for="" i="" j="">/* delay loop */
A few notes should be mentioned about this program:
- Note that the main program is also a thread, so it executes the
do_loop()function in parallel to the thread it creates. pthread_create()gets 4 parameters. The first parameter is used bypthread_create()to supply the program with information about the thread. The second parameter is used to set some attributes for the new thread. In our case we supplied a NULL pointer to tellpthread_create()to use the default values. The third parameter is the name of the function that the thread will start executing. The forth parameter is an argument to pass to this function. Note the cast to a 'void*'. It is not required by ANSI-C syntax, but is placed here for clarification.- The delay loop inside the function is used only to demonstrate that the threads are executing in parallel. Use a larger delay value if your CPU runs too fast, and you see all the printouts of one thread before the other.
- The call to
pthread_exit()Causes the current thread to exit and free any thread-specific resources it is taking. There is no need to use this call at the end of the thread's top function, since when it returns, the thread would exit automatically anyway. This function is useful if we want to exit a thread in the middle of its execution.
In order to compile a multi-threaded program using
The source code for this program may be found in the pthread_create.c file.
gcc, we need to link it with the pthreads library. Assuming you have this library already installed on your system, here is how to compile our first program:gcc pthread_create.c -o pthread_create -lpthread The source code for this program may be found in the pthread_create.c file.
Synchronizing Threads With Mutexes
One of the basic problems when running several threads that use the same memory space, is making sure they don't "step on each other's toes". By this we refer to the problem of using a data structure from two different threads.
For instance, consider the case where two threads try to update two variables. One tries to set both to 0, and the other tries to set both to 1. If both threads would try to do that at the same time, we might get with a situation where one variable contains 1, and one contains 0. This is because a context-switch (we already know what this is by now, right?) might occur after the first tread zeroed out the first variable, then the second thread would set both variables to 1, and when the first thread resumes operation, it will zero out the second variable, thus getting the first variable set to '1', and the second set to '0'.
What Is A Mutex?
A basic mechanism supplied by the pthreads library to solve this problem, is called a mutex. A mutex is a lock that guarantees three things:
- Atomicity - Locking a mutex is an atomic operation, meaning that the operating system (or threads library) assures you that if you locked a mutex, no other thread succeeded in locking this mutex at the same time.
- Singularity - If a thread managed to lock a mutex, it is assured that no other thread will be able to lock the thread until the original thread releases the lock.
- Non-Busy Wait - If a thread attempts to lock a thread that was locked by a second thread, the first thread will be suspended (and will not consume any CPU resources) until the lock is freed by the second thread. At this time, the first thread will wake up and continue execution, having the mutex locked by it.
From these three points we can see how a mutex can be used to assure exclusive access to variables (or in general critical code sections). Here is some pseudo-code that updates the two variables we were talking about in the previous section, and can be used by the first thread:
lock mutex 'X1'. set first variable to '0'. set second variable to '0'. unlock mutex 'X1'.
Meanwhile, the second thread will do something like this:
lock mutex 'X1'. set first variable to '1'. set second variable to '1'. unlock mutex 'X1'.
Assuming both threads use the same mutex, we are assured that after they both ran through this code, either both variables are set to '0', or both are set to '1'. You'd note this requires some work from the programmer - If a third thread was to access these variables via some code that does not use this mutex, it still might mess up the variable's contents. Thus, it is important to enclose all the code that accesses these variables in a small set of functions, and always use only these functions to access these variables.
Creating And Initializing A Mutex
In order to create a mutex, we first need to declare a variable of typepthread_mutex_t , and then initialize it. The simplest way it by assigning it the PTHREAD_MUTEX_INITIALIZER constant. So we'll use a code that looks something like this:
pthread_mutex_t a_mutex = PTHREAD_MUTEX_INITIALIZER;
One note should be made here: This type of initialization creates a mutex called 'fast mutex'. This means that if a thread locks the mutex and then tries to lock it again, it'll get stuck - it will be in a deadlock.
There is another type of mutex, called 'recursive mutex', which allows the thread that locked it, to lock it several more times, without getting blocked (but other threads that try to lock the mutex now will get blocked). If the thread then unlocks the mutex, it'll still be locked, until it is unlocked the same amount of times as it was locked. This is similar to the way modern door locks work - if you turned it twice clockwise to lock it, you need to turn it twice counter-clockwise to unlock it. This kind of mutex can be created by assigning the constant
PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP to a mutex variable.Locking And Unlocking A Mutex
In order to lock a mutex, we may use the functionpthread_mutex_lock(). This function attempts to lock the mutex, or block the thread if the mutex is already locked by another thread. In this case, when the mutex is unlocked by the first process, the function will return with the mutex locked by our process. Here is how to lock a mutex (assuming it was initialized earlier):
int rc = pthread_mutex_lock(&a_mutex);
if (rc) { /* an error has occurred */
perror("pthread_mutex_lock");
pthread_exit(NULL);
}
/* mutex is now locked - do your stuff. */
.
.
After the thread did what it had to (change variables or data structures, handle file, or whatever it intended to do), it should free the mutex, using the
pthread_mutex_unlock() function, like this:
rc = pthread_mutex_unlock(&a_mutex);
if (rc) {
perror("pthread_mutex_unlock");
pthread_exit(NULL);
}
Destroying A Mutex
After we finished using a mutex, we should destroy it. Finished using means no thread needs it at all. If only one thread finished with the mutex, it should leave it alive, for the other threads that might still need to use it. Once all finished using it, the last one can destroy it using thepthread_mutex_destroy() function:
rc = pthread_mutex_destroy(&a_mutex);
After this call, this variable (a_mutex) may not be used as a mutex any more, unless it is initialized again. Thus, if one destroys a mutex too early, and another thread tries to lock or unlock it, that thread will get a
EINVAL error code from the lock or unlock function.Using A Mutex - A Complete Example
After we have seen the full life cycle of a mutex, lets see an example program that uses a mutex. The program introduces two employees competing for the "employee of the day" title, and the glory that comes with it. To simulate that in a rapid pace, the program employs 3 threads: one that promotes Danny to "employee of the day", one that promotes Moshe to that situation, and a third thread that makes sure that the employee of the day's contents is consistent (i.e. contains exactly the data of one employee).Two copies of the program are supplied. One that uses a mutex, and one that does not. Try them both, to see the differences, and be convinced that mutexes are essential in a multi-threaded environment.
The programs themselves are in the files accompanying this tutorial. The one that uses a mutex is employee-with-mutex.c. The one that does not use a mutex is employee-without-mutex.c. Read the comments inside the source files to get a better understanding of how they work.
Starvation And Deadlock Situations
pthread_mutex_lock() might block for a non-determined duration, in case of the mutex being already locked. If it remains locked forever, it is said that our poor thread is "starved" - it was trying to acquire a resource, but never got it. It is up to the programmer to ensure that such starvation won't occur. The pthread library does not help us with that.The pthread library might, however, figure out a "deadlock". A deadlock is a situation in which a set of threads are all waiting for resources taken by other threads, all in the same set. Naturally, if all threads are blocked waiting for a mutex, none of them will ever come back to life again. The pthread library keeps track of such situations, and thus would fail the last thread trying to call
pthread_mutex_lock(), with an error of type EDEADLK. The programmer should check for such a value, and take steps to solve the deadlock somehow.Refined Synchronization - Condition Variables
As we've seen before with mutexes, they allow for simple coordination - exclusive access to a resource. However, we often need to be able to make real synchronization between threads:- In a server, one thread reads requests from clients, and dispatches them to several threads for handling. These threads need to be notified when there is data to process, otherwise they should wait without consuming CPU time.
- In a GUI (Graphical User Interface) Application, one thread reads user input, another handles graphical output, and a third thread sends requests to a server and handles its replies. The server-handling thread needs to be able to notify the graphics-drawing thread when a reply from the server arrived, so it will immediately show it to the user. The user-input thread needs to be always responsive to the user, for example, to allow her to cancel long operations currently executed by the server-handling thread.
What Is A Condition Variable?
A condition variable is a mechanism that allows threads to wait (without wasting CPU cycles) for some even to occur. Several threads may wait on a condition variable, until some other thread signals this condition variable (thus sending a notification). At this time, one of the threads waiting on this condition variable wakes up, and can act on the event. It is possible to also wake up all threads waiting on this condition variable by using a broadcast method on this variable.Note that a condition variable does not provide locking. Thus, a mutex is used along with the condition variable, to provide the necessary locking when accessing this condition variable.
Creating And Initializing A Condition Variable
Creation of a condition variable requires defining a variable of typepthread_cond_t, and initializing it properly. Initialization may be done with either a simple use of a macro named PTHREAD_COND_INITIALIZER or the usage of the pthread_cond_init() function. We will show the first form here:pthread_cond_t got_request = PTHREAD_COND_INITIALIZER; This defines a condition variable named 'got_request', and initializes it.
Note: since the
PTHREAD_COND_INITIALIZER is actually a structure, it may be used to initialize a condition variable only when it is declared. In order to initialize it during runtime, one must use the pthread_cond_init() function.Signaling A Condition Variable
In order to signal a condition variable, one should either thepthread_cond_signal() function (to wake up a only one thread waiting on this variable), or the pthread_cond_broadcast() function (to wake up all threads waiting on this variable). Here is an example using signal, assuming 'got_request' is a properly initialized condition variable:int rc = pthread_cond_signal(&got_request); Or by using the broadcast function:
int rc = pthread_cond_broadcast(&got_request); When either function returns, 'rc' is set to 0 on success, and to a non-zero value on failure. In such a case (failure), the return value denotes the error that occured (
EINVAL denotes that the given parameter is not a condition variable. ENOMEM denotes that the system has run out of memory.Note: success of a signaling operation does not mean any thread was awakened - it might be that no thread was waiting on the condition variable, and thus the signaling does nothing (i.e. the signal is lost).
It is also not remembered for future use - if after the signaling function returns another thread starts waiting on this condition variable, a further signal is required to wake it up.
Waiting On A Condition Variable
If one thread signals the condition variable, other threads would probably want to wait for this signal. They may do so using one of two functions,pthread_cond_wait() or pthread_cond_timedwait(). Each of these functions takes a condition variable, and a mutex (which should be locked before calling the wait function), unlocks the mutex, and waits until the condition variable is signaled, suspending the thread's execution. If this signaling causes the thread to awake (see discussion of pthread_cond_signal() earlier), the mutex is automagically locked again by the wait funciton, and the wait function returns.pthread_cond_timedwait() allows the programmer to specify a timeout for the waiting, after which the function always returns, with a proper error value (ETIMEDOUT) to notify that condition variable was NOT signaled before the timeout passed. The pthread_cond_wait() would wait indefinitely if it was never signaled.Here is how to use these two functions. We make the assumption that 'got_request' is a properly initialized condition variable, and that 'request_mutex' is a properly initialized mutex. First, we try the
pthread_cond_wait() function:
/* first, lock the mutex */
int rc = pthread_mutex_lock(&a_mutex);
if (rc) { /* an error has occurred */
perror("pthread_mutex_lock");
pthread_exit(NULL);
}
/* mutex is now locked - wait on the condition variable. */
/* During the execution of pthread_cond_wait, the mutex is unlocked. */
rc = pthread_cond_wait(&got_request, &a_mutex);
if (rc == 0) { /* we were awakened due to the cond. variable being signaled */
/* The mutex is now locked again by pthread_cond_wait() */
/* do your stuff... */
.
}
/* finally, unlock the mutex */
pthread_mutex_unlock(&a_mutex);
Now an example using the
pthread_cond_timedwait() function:
#include /* struct timeval definition */
#include /* declaration of gettimeofday() */
struct timeval now; /* time when we started waiting */
struct timespec timeout; /* timeout value for the wait function */
int done; /* are we done waiting? */
/* first, lock the mutex */
int rc = pthread_mutex_lock(&a_mutex);
if (rc) { /* an error has occurred */
perror("pthread_mutex_lock");
pthread_exit(NULL);
}
/* mutex is now locked */
/* get current time */
gettimeofday(&now);
/* prepare timeout value */
timeout.tv_sec = now.tv_sec + 5
timeout.tv_nsec = now.tv_usec * 1000; /* timeval uses microseconds. */
/* timespec uses nanoseconds. */
/* 1 nanosecond = 1000 micro seconds. */
/* wait on the condition variable. */
/* we use a loop, since a Unix signal might stop the wait before the timeout */
done = 0;
while (!done) {
/* remember that pthread_cond_timedwait() unlocks the mutex on entrance */
rc = pthread_cond_timedwait(&got_request, &a_mutex, &timeout);
switch(rc) {
case 0: /* we were awakened due to the cond. variable being signaled */
/* the mutex was now locked again by pthread_cond_timedwait. */
/* do your stuff here... */
.
.
done = 0;
break;
case ETIMEDOUT: /* our time is up */
done = 0;
break;
default: /* some error occurred (e.g. we got a Unix signal) */
break; /* break this switch, but re-do the while loop. */
}
}
/* finally, unlock the mutex */
pthread_mutex_unlock(&a_mutex);
As you can see, the timed wait version is way more complex, and thus better be wrapped up by some function, rather then being re-coded in every necessary location.
Note: it might be that a condition variable that has 2 or more threads waiting on it is signaled many times, and yet one of the threads waiting on it never awakened. This is because we are not guaranteed which of the waiting threads is awakened when the variable is signaled. It might be that the awakened thread quickly comes back to waiting on the condition variables, and gets awakened again when the variable is signaled again, and so on. The situation for the un-awakened thread is called 'starvation'. It is up to the programmer to make sure this situation does not occur if it implies bad behavior. Yet, in our server example from before, this situation might indicate requests are coming in a very slow pace, and thus perhaps we have too many threads waiting to service requests. In this case, this situation is actually good, as it means every request is handled immediately when it arrives.
Note 2: when the mutex is being broadcast (using pthread_cond_broadcast), this does not mean all threads are running together. Each of them tries to lock the mutex again before returning from their wait function, and thus they'll start running one by one, each one locking the mutex, doing their work, and freeing the mutex before the next thread gets its chance to run.
Destroying A Condition Variable
After we are done using a condition variable, we should destroy it, to free any system resources it might be using. This can be done using thepthread_cond_destroy(). In order for this to work, there should be no threads waiting on this condition variable. Here is how to use this function, again, assuming 'got_request' is a pre-initialized condition variable:
int rc = pthread_cond_destroy(&got_request);
if (rc == EBUSY) { /* some thread is still waiting on this condition variable */
/* handle this case here... */
.
.
}
What if some thread is still waiting on this variable? depending on the case, it might imply some flaw in the usage of this variable, or just lack of proper thread cleanup code. It is probably good to alert the programmer, at least during debug phase of the program, of such a case. It might mean nothing, but it might be significant.
A Real Condition For A Condition Variable
A note should be taken about condition variables - they are usually pointless without some real condition checking combined with them. To make this clear, lets consider the server example we introduced earlier. Assume that we use the 'got_request' condition variable to signal that a new request has arrived that needs handling, and is held in some requests queue. If we had threads waiting on the condition variable when this variable is signaled, we are assured that one of these threads will awake and handle this request.However, what if all threads are busy handling previous requests, when a new one arrives? the signaling of the condition variable will do nothing (since all threads are busy doing other things, NOT waiting on the condition variable now), and after all threads finish handling their current request, they come back to wait on the variable, which won't necessarily be signaled again (for example, if no new requests arrive). Thus, there is at least one request pending, while all handling threads are blocked, waiting for a signal.
In order to overcome this problem, we may set some integer variable to denote the number of pending requests, and have each thread check the value of this variable before waiting on the variable. If this variable's value is positive, some request is pending, and the thread should go and handle it, instead of going to sleep. Further more, a thread that handled a request, should reduce the value of this variable by one, to make the count correct.
Lets see how this affects the waiting code we have seen above.
/* number of pending requests, initially none */
int num_requests = 0;
.
.
/* first, lock the mutex */
int rc = pthread_mutex_lock(&a_mutex);
if (rc) { /* an error has occurred */
perror("pthread_mutex_lock");
pthread_exit(NULL);
}
/* mutex is now locked - wait on the condition variable */
/* if there are no requests to be handled. */
rc = 0;
if (num_requests == 0)
rc = pthread_cond_wait(&got_request, &a_mutex);
if (num_requests > 0 && rc == 0) { /* we have a request pending */
/* do your stuff... */
.
.
/* decrease count of pending requests */
num_requests--;
}
}
/* finally, unlock the mutex */
pthread_mutex_unlock(&a_mutex);
Full Article here
http://www.cs.kent.edu/~ruttan/sysprog/lectures/multi-thread/multi-thread.htmlWednesday, June 20, 2007
Listing All Registered ActiveXControls / COM object based on there category
//Initialise COM libraries
CoInitialize (NULL);
//The Component Category Manager implemented by System implements
//this interface
ICatInformation *pCatInfo=NULL;
//Create an instance of standard Component Category Manager
HRESULT hr=CoCreateInstance (CLSID_StdComponentCategoriesMgr ,
NULL,
CLSCTX_INPROC_SERVER,
IID_ICatInformation ,
(void **)&pCatInfo);
//Increase ref count on interface
pCatInfo->AddRef ();
//IEnumGUID interface provides enumerator for enumerating through
//the collection of COM objects
IEnumGUID *pEnumGUID=NULL;
//We are intersted in finding out only controls so put CATID_Control
//in the array
CATID pcatidImpl[1];
CATID pcatidReqd[1];
// Want only my Plugin Category
pcatidImpl[0]=CATID_MyPlugin;
pcatidReqd[1]=CATID_MyPlugin;
// Want all Active X Control
//pcatidImpl[0]=CATID_Control;
//Now enumerate the classes i.e. COM objects of this type.
pCatInfo->EnumClassesOfCategories (1,
pcatidImpl,
0,
pcatidReqd ,
&pEnumGUID);
//Enumerate as long as you get S_OK
CLSID clsid;
while( (hr= pEnumGUID->Next( 1, &clsid, NULL ))==S_OK )
{
BSTR bstrClassName; //Get the information of class
//This is what MSDN says about the parameters
/*-----------------------------------------------
USERCLASSTYPE_FULL The full type name of the class.
USERCLASSTYPE_SHORT A short name (maximum of 15 characters) that
is used for popup menus and the Links dialog
box.
USERCLASSTYPE_APPNAME The name of the application servicing the class
and is used in the Result text in dialog boxes.
-----------------------------------------------*/
OleRegGetUserType (clsid,USERCLASSTYPE_FULL,&bstrClassName);
CString strControlName(bstrClassName);
//Add string in our listbox
m_list1.AddString (strControlName);
}
//we are done so now release the interface ptr
pCatInfo->Release ();
CoUninitialize ();
CoInitialize (NULL);
//The Component Category Manager implemented by System implements
//this interface
ICatInformation *pCatInfo=NULL;
//Create an instance of standard Component Category Manager
HRESULT hr=CoCreateInstance (CLSID_StdComponentCategoriesMgr ,
NULL,
CLSCTX_INPROC_SERVER,
IID_ICatInformation ,
(void **)&pCatInfo);
//Increase ref count on interface
pCatInfo->AddRef ();
//IEnumGUID interface provides enumerator for enumerating through
//the collection of COM objects
IEnumGUID *pEnumGUID=NULL;
//We are intersted in finding out only controls so put CATID_Control
//in the array
CATID pcatidImpl[1];
CATID pcatidReqd[1];
// Want only my Plugin Category
pcatidImpl[0]=CATID_MyPlugin;
pcatidReqd[1]=CATID_MyPlugin;
// Want all Active X Control
//pcatidImpl[0]=CATID_Control;
//Now enumerate the classes i.e. COM objects of this type.
pCatInfo->EnumClassesOfCategories (1,
pcatidImpl,
0,
pcatidReqd ,
&pEnumGUID);
//Enumerate as long as you get S_OK
CLSID clsid;
while( (hr= pEnumGUID->Next( 1, &clsid, NULL ))==S_OK )
{
BSTR bstrClassName; //Get the information of class
//This is what MSDN says about the parameters
/*-----------------------------------------------
USERCLASSTYPE_FULL The full type name of the class.
USERCLASSTYPE_SHORT A short name (maximum of 15 characters) that
is used for popup menus and the Links dialog
box.
USERCLASSTYPE_APPNAME The name of the application servicing the class
and is used in the Result text in dialog boxes.
-----------------------------------------------*/
OleRegGetUserType (clsid,USERCLASSTYPE_FULL,&bstrClassName);
CString strControlName(bstrClassName);
//Add string in our listbox
m_list1.AddString (strControlName);
}
//we are done so now release the interface ptr
pCatInfo->Release ();
CoUninitialize ();
Waiting for the Prj2make# Visual Studio Add-in For Mono
Right now Novell provide a command line utility to migrate a Microsoft Prj
to Mono.
http://forge.novell.com/modules/xfcontent/downloads.php/prj2make-sharp
Looking forward to see this project.
I have completed a Visual Studio Doxygen Add On if the author want
to contact me.
http://www.mfconsulting.com/product/gtks-inst4win/vsprj2make_Proposal.html
to Mono.
http://forge.novell.com/modules/xfcontent/downloads.php/prj2make-sharp
Looking forward to see this project.
I have completed a Visual Studio Doxygen Add On if the author want
to contact me.
http://www.mfconsulting.com/product/gtks-inst4win/vsprj2make_Proposal.html
Tuesday, June 19, 2007
java to c# and c# to java
Microsoft has the Java Language Conversion Assistant (JLCA) that enables
developers to migrate their existing Java code to C# on the .NET Framework.
It can be found at:
http://msdn2.microsoft.com/en-us/vstudio/aa718346.aspx
However, the reverse, a C# to Java is more difficlut to achieve
http://jsc.sourceforge.net/
developers to migrate their existing Java code to C# on the .NET Framework.
It can be found at:
http://msdn2.microsoft.com/en-us/vstudio/aa718346.aspx
However, the reverse, a C# to Java is more difficlut to achieve
http://jsc.sourceforge.net/
Thursday, June 14, 2007
Direct Show Filters Rules
APE Source Filter: DirectShow audio decoder filter used to decode Monkey's Audio files (APE files)
AC3 Filter: DirectShow audio decoder and processor filter used to decode audio tracks in movies (DVD, MPEG4 and others)
AVI Splitter: Replacement for DirectShow's default Audio Video Interleave (AVI) Splitter (It can open many broken files and can reindex when needed)
CDDA Reader Filter: DirectShow audio decoder capable of reading Audio CDs
CDXA Reader Filter: DirectShow video decoder capable of reading (S)VCDs and XCDs
CoreAAC Filter: Advanced Audio Coding (AAC) DirectShow audio decoder capable of reading AAC files
CoreFlac Decoder: DirectShow audio decoder for the Flac audio codec
FFDShow: DirectShow decoding filter for decompressing DivX, XviD, H.264, FLV1, WMV, MPEG-1 and MPEG-2, MPEG-4 and various other audio and video formats. It uses libavcodec from ffmpeg project for video decompression, postprocessing code from mplayer to enhance visual quality of low bitrate movies, and is based on original DirectShow filter from XviD, which is GPL'ed educational implementation of MPEG4 encoder
FLV Splitter: Capable of splitting Flash Video files (FLV) and decoding On2 VP62 Video streams (notable users of the FLV format include YouTube, Google Video, Reuters.com, Yahoo! Video and MySpace.)
OGG Splitter: DirectShow OGG Splitter capable of splitting OGG and OGM file(s) and enables to play / create .ogm wrapped video
GPL MPEG-1/2 Decoder: DirectShow MPEG decoder filter can be used to play MPEG-1 and MPEG-2 streams
MOD Source Filter: DirectShow filter decoder capable of decoding Tracker files (MO3, IT, XM, S3M, MTM, MOD and UMX)
MPEG Splitter: DirectShow splitter capable of splitting MPEG1/MPEG2 files (MP(E)G, VOB, DAT, TS, TP, etc.)
MPV Decoder: DirectShow decoder capable of decoding MPEG1/MPEG2 Video streams, and DVDs
MPA Splitter: DirectShow Splitter capable of splitting MPEG Audio files (MP3) and MPEG4 Audio files (AAC)
MPC Source Filter: DirectShow decoder filter capable of decoding MusePack Audio files (MPC).
MP4 Splitter: DirectShow Splitter capable of splitting MP4, M4A, 3GP and HD-MOV files
Matroska Splitter: DirectShow splitter capable of splitting Matroska files (MKV and MKA)
Media Player Classic: an open source media player that looks just like Windows Media Player, but has many additional features. It has a built in DVD player with real-time zoom, support for AVI subtitles, QuickTime and RealVideo support
OFR Source Filter: DirectShow filter decoder capable of decoding OptimFROG Audio files (OFR)
RealMedia Splitter: DirectShow splitter that allows you to play RealNetworks audio/video files.
SHOUTcast Source Filter: DirectShow decoder filter capable of handling internet radio stations streams (http://www.shoutcast.com)
WV Splitter / Decoder: DirectShow WV Splitter and decoder capable of splitting and decoding WavPack Audio files (WV)
VobSub Filter: DirectShow Vob Subtitles filter capable of displaying (embedded) Subtitle files (SRT, SUB, IDX, etc)
AC3 Filter: DirectShow audio decoder and processor filter used to decode audio tracks in movies (DVD, MPEG4 and others)
AVI Splitter: Replacement for DirectShow's default Audio Video Interleave (AVI) Splitter (It can open many broken files and can reindex when needed)
CDDA Reader Filter: DirectShow audio decoder capable of reading Audio CDs
CDXA Reader Filter: DirectShow video decoder capable of reading (S)VCDs and XCDs
CoreAAC Filter: Advanced Audio Coding (AAC) DirectShow audio decoder capable of reading AAC files
CoreFlac Decoder: DirectShow audio decoder for the Flac audio codec
FFDShow: DirectShow decoding filter for decompressing DivX, XviD, H.264, FLV1, WMV, MPEG-1 and MPEG-2, MPEG-4 and various other audio and video formats. It uses libavcodec from ffmpeg project for video decompression, postprocessing code from mplayer to enhance visual quality of low bitrate movies, and is based on original DirectShow filter from XviD, which is GPL'ed educational implementation of MPEG4 encoder
FLV Splitter: Capable of splitting Flash Video files (FLV) and decoding On2 VP62 Video streams (notable users of the FLV format include YouTube, Google Video, Reuters.com, Yahoo! Video and MySpace.)
OGG Splitter: DirectShow OGG Splitter capable of splitting OGG and OGM file(s) and enables to play / create .ogm wrapped video
GPL MPEG-1/2 Decoder: DirectShow MPEG decoder filter can be used to play MPEG-1 and MPEG-2 streams
MOD Source Filter: DirectShow filter decoder capable of decoding Tracker files (MO3, IT, XM, S3M, MTM, MOD and UMX)
MPEG Splitter: DirectShow splitter capable of splitting MPEG1/MPEG2 files (MP(E)G, VOB, DAT, TS, TP, etc.)
MPV Decoder: DirectShow decoder capable of decoding MPEG1/MPEG2 Video streams, and DVDs
MPA Splitter: DirectShow Splitter capable of splitting MPEG Audio files (MP3) and MPEG4 Audio files (AAC)
MPC Source Filter: DirectShow decoder filter capable of decoding MusePack Audio files (MPC).
MP4 Splitter: DirectShow Splitter capable of splitting MP4, M4A, 3GP and HD-MOV files
Matroska Splitter: DirectShow splitter capable of splitting Matroska files (MKV and MKA)
Media Player Classic: an open source media player that looks just like Windows Media Player, but has many additional features. It has a built in DVD player with real-time zoom, support for AVI subtitles, QuickTime and RealVideo support
OFR Source Filter: DirectShow filter decoder capable of decoding OptimFROG Audio files (OFR)
RealMedia Splitter: DirectShow splitter that allows you to play RealNetworks audio/video files.
SHOUTcast Source Filter: DirectShow decoder filter capable of handling internet radio stations streams (http://www.shoutcast.com)
WV Splitter / Decoder: DirectShow WV Splitter and decoder capable of splitting and decoding WavPack Audio files (WV)
VobSub Filter: DirectShow Vob Subtitles filter capable of displaying (embedded) Subtitle files (SRT, SUB, IDX, etc)
Friday, June 08, 2007
Manifest hell
I thought Manifests were an end to DLL hell - but it just seems that now we have manifest hell
A solution to two references to different versions of CRT, MFC, ATL in one application manifest file
I have received several questions about a case when developers find two or more references to different versions of CRT or MFC or ALT libraries in application manifest. Usually application manifest would look similar to the following:
Notice underlined version of assembly. This manifest tells Windows loader to load two different versions of the same assembly. The loader is going to do what it is told to. But the problem arises when it can only find one version (50608) of them. There are two cases when this may happen:
1) The oldest version of VC++ library is installed in WinSxS folder. For example, author of the application may have redistributed VS2005 RTM version of CRT library using MSMs or VCRedist.EXE, but ones parts of application were rebuilt with VS2005 SP1 and deployed to the same machine, SP1 versions of MSMs or VCRedist.EXE were not deployed to the same machine.
2) It is not possible to have two versions of a private assembly in one folder. For example, an author of an application has deployed the RTM version of CRT library as a private assembly in application’s local folder. After application is rebuilt with SP1, the author faces a challenge of having two copies of same files/folders in application local folder. If she keeps RTM version, Windows loader is going to complain about SP1 version missing. If she keeps SP1 version, Windows loaders is going to fail to find RTM version. Windows shows a message box which says :”The application has failed to start because of side-by-side configuration is incorrect,” and event viewer shows error details similar to “Activation context generation failed for "…\foo.exe".Error in manifest or policy file. A component version required by the application conflicts with another component version already active.” Bottom line, application does not start.
The root cause of the issue is that not all parts of application’s source code are built with the same version of VC++ libraries and tools. The linker catches some cases and reports errors when it tries to link objects and libraries built with different versions of compiler. However it is still possible to get pass the linker and an application may get these two dependencies in its manifest.
There is one solution to the problem and two workarounds.
The solution. To fix the problem, you must rebuild all parts of your code with the newest toolset and libraries. Make sure you have cleaned up all binaries built by the old toolset and started full rebuild of the whole source base. Also check that old versions of headers and import libraries for VC++ libraries are not on INCLUDE and LIB path and they are not used during the build.
If you cannot rebuild all your code and use one version of VC libraries, there are two ways to work around this problem:
Workaround#1: Install the newer version (8.0.50727.762 in this case) of VC++ MSMs or VCRedist.EXE on a machine where your application is going to run. Once policy for VC++ assemblies is installed on that machine they are going to redirect all loads of older versions (8.0.50608.0) to the newest version available on the machine.
Workaround#2: If you are redistributing VC++ libraries in application’s local folder, you need to add an application configuration file that redirects an attempt to load 8.0.50608.0 version to 8.0.50727.762 version. Configuration file has to have same name as the exe plus .config extension and has to be right next to exe or embedded into the EXE. Here is an example of a configuration file that one would use to resolve issue with the manifest from above:
inferior and superior char have been replace by { and } because google blogs text editor is retarded
{configuration}
{windows}
{assemblybinding xmlns="urn:schemas-microsoft-com:asm.v1"}
{dependentassembly}
{assemblyidentity publickeytoken="1fc8b3b9a1e18e3b" processorarchitecture="x86" name="Microsoft.VC80.CRT" type="win32"}{/assemblyidentity}
{bindingredirect newversion="8.0.50727.762" oldversion="8.0.41204.256-8.0.50727.762"}
{/dependentassembly}
{/assemblybinding}
{/windows}
{/configuration}
This file basically redirects attempts to load any version of VC CRT greater or equal to 8.0.41204.256 (VS2005 Beta 1) and less than 8.0.50727.762(VS2005 SP1) to load VS2005 SP1 version of VC CRT. It is similar to what the policy file does for a case when CRT is installed into WinSxS folder.
Overall my recommendation is to never use static libraries for which you do not have a source code. Using a static library without its source always puts a set of restrictions on build configuration of a binary that consumes it. There are ways to build a “clean” static library, but they are not well known among developers and it is very rare to find a static library that can be consumed in a code built with different versions of compiler, linker and libraries. If source code is not provided, I would only take a dependency on a DLL and only if its APIs are cleanly designed to obey rules of data exchange across DLL boundary. COM and .Net class assemblies are designed to address this problem and they are the easiest technologies to use in this scenario.
http://msdn2.microsoft.com/en-us/library/aa374182.aspx
Visual Studio 2005 SP1
http://www.microsoft.com/downloads/thankyou.aspx?familyId=bb4a75ab-e2d4-4c96-b39d-37baf6b5b1dc&displayLang=en
VCRedist x86
http://www.microsoft.com/downloads/details.aspx?familyid=32bc1bee-a3f9-4c13-9c99-220b62a191ee&displaylang=en
Visual Studio 2005 SP1
Installation fails on Windows Server 2003 editions with Windows Server 2003 SP1 Installed:
loadTOCNode(3, 'moreinformation');
The error reported is
Error 1718. File Filename was rejected by digital signature policy.This problem occurs when the computer has insufficient contiguous memory for Windows Server 2003 or Windows XP to verify that the .msi package or the .msp package is correctly signed.To resolve this issue:
•
Refer to KB article 925336 (http://support.microsoft.com/kb/925336).
VS2005 SP1 and Microsoft.VC80.CRT in applocal mode
problem running some of our applications in "applocal" mode since we have upgraded to SP1.
We use some external libraries (we don't have the source code) that were compiled with the original VS2005.
The following manifest is therefore generated:
The application runs fine when the CRT in present in WinSxS (the Policy does the correct redirection). However, if you want to use the "applocal" mode for this particular application.
When running "app.exe" if you get this message: "This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.
Choose external manifest
_USE_RTM_VERSION is newly introduced in SP1 (and previously also in hotfix 919280). It is found in crtassem.h and crtdefs.h. If you use _USE_RTM_VERSION - it must be done in all EXEs and DLLs the app uses. The _USE_RTM_VERSION define determines which CRT version is placed in your manifest file. If it is defined, then the old version number is put in.
If not defined, then it puts the new version in (what ever version ends up shipping with SP1)And no, using _USE_RTM_VERSION does not avoid the runtime rebinding to the new version. The fact is, it only gives the apps the ability to use the originals, but if the new ones are there on the machine in WinSxS, then it uses the new ones instead. Without _USE_RTM_VERSION, if the originals are there and the new ones aren't, then the app doesn't even run.Without using WinSxS, even if you install applocal and if same or newer versions exist in WinSxS, then those in WinSxS will be used instead. However, there is one way to trick the system into never using the new ones from WinSxS, and only using your applocal (original) ones.
For more info see http://blog.kalmbachnet.de/?postid=80
I know of no way to force an app to use a specific version out of WinSxS, if a redirection policy has been installed.
A solution to two references to different versions of CRT, MFC, ATL in one application manifest file
I have received several questions about a case when developers find two or more references to different versions of CRT or MFC or ALT libraries in application manifest. Usually application manifest would look similar to the following:
Notice underlined version of assembly. This manifest tells Windows loader to load two different versions of the same assembly. The loader is going to do what it is told to. But the problem arises when it can only find one version (50608) of them. There are two cases when this may happen:
1) The oldest version of VC++ library is installed in WinSxS folder. For example, author of the application may have redistributed VS2005 RTM version of CRT library using MSMs or VCRedist.EXE, but ones parts of application were rebuilt with VS2005 SP1 and deployed to the same machine, SP1 versions of MSMs or VCRedist.EXE were not deployed to the same machine.
2) It is not possible to have two versions of a private assembly in one folder. For example, an author of an application has deployed the RTM version of CRT library as a private assembly in application’s local folder. After application is rebuilt with SP1, the author faces a challenge of having two copies of same files/folders in application local folder. If she keeps RTM version, Windows loader is going to complain about SP1 version missing. If she keeps SP1 version, Windows loaders is going to fail to find RTM version. Windows shows a message box which says :”The application has failed to start because of side-by-side configuration is incorrect,” and event viewer shows error details similar to “Activation context generation failed for "…\foo.exe".Error in manifest or policy file. A component version required by the application conflicts with another component version already active.” Bottom line, application does not start.
The root cause of the issue is that not all parts of application’s source code are built with the same version of VC++ libraries and tools. The linker catches some cases and reports errors when it tries to link objects and libraries built with different versions of compiler. However it is still possible to get pass the linker and an application may get these two dependencies in its manifest.
There is one solution to the problem and two workarounds.
The solution. To fix the problem, you must rebuild all parts of your code with the newest toolset and libraries. Make sure you have cleaned up all binaries built by the old toolset and started full rebuild of the whole source base. Also check that old versions of headers and import libraries for VC++ libraries are not on INCLUDE and LIB path and they are not used during the build.
If you cannot rebuild all your code and use one version of VC libraries, there are two ways to work around this problem:
Workaround#1: Install the newer version (8.0.50727.762 in this case) of VC++ MSMs or VCRedist.EXE on a machine where your application is going to run. Once policy for VC++ assemblies is installed on that machine they are going to redirect all loads of older versions (8.0.50608.0) to the newest version available on the machine.
Workaround#2: If you are redistributing VC++ libraries in application’s local folder, you need to add an application configuration file that redirects an attempt to load 8.0.50608.0 version to 8.0.50727.762 version. Configuration file has to have same name as the exe plus .config extension and has to be right next to exe or embedded into the EXE. Here is an example of a configuration file that one would use to resolve issue with the manifest from above:
inferior and superior char have been replace by { and } because google blogs text editor is retarded
{configuration}
{windows}
{assemblybinding xmlns="urn:schemas-microsoft-com:asm.v1"}
{dependentassembly}
{assemblyidentity publickeytoken="1fc8b3b9a1e18e3b" processorarchitecture="x86" name="Microsoft.VC80.CRT" type="win32"}{/assemblyidentity}
{bindingredirect newversion="8.0.50727.762" oldversion="8.0.41204.256-8.0.50727.762"}
{/dependentassembly}
{/assemblybinding}
{/windows}
{/configuration}
This file basically redirects attempts to load any version of VC CRT greater or equal to 8.0.41204.256 (VS2005 Beta 1) and less than 8.0.50727.762(VS2005 SP1) to load VS2005 SP1 version of VC CRT. It is similar to what the policy file does for a case when CRT is installed into WinSxS folder.
Overall my recommendation is to never use static libraries for which you do not have a source code. Using a static library without its source always puts a set of restrictions on build configuration of a binary that consumes it. There are ways to build a “clean” static library, but they are not well known among developers and it is very rare to find a static library that can be consumed in a code built with different versions of compiler, linker and libraries. If source code is not provided, I would only take a dependency on a DLL and only if its APIs are cleanly designed to obey rules of data exchange across DLL boundary. COM and .Net class assemblies are designed to address this problem and they are the easiest technologies to use in this scenario.
http://msdn2.microsoft.com/en-us/library/aa374182.aspx
Visual Studio 2005 SP1
http://www.microsoft.com/downloads/thankyou.aspx?familyId=bb4a75ab-e2d4-4c96-b39d-37baf6b5b1dc&displayLang=en
VCRedist x86
http://www.microsoft.com/downloads/details.aspx?familyid=32bc1bee-a3f9-4c13-9c99-220b62a191ee&displaylang=en
Visual Studio 2005 SP1
Installation fails on Windows Server 2003 editions with Windows Server 2003 SP1 Installed:
loadTOCNode(3, 'moreinformation');
The error reported is
Error 1718. File Filename was rejected by digital signature policy.This problem occurs when the computer has insufficient contiguous memory for Windows Server 2003 or Windows XP to verify that the .msi package or the .msp package is correctly signed.To resolve this issue:
•
Refer to KB article 925336 (http://support.microsoft.com/kb/925336).
VS2005 SP1 and Microsoft.VC80.CRT in applocal mode
problem running some of our applications in "applocal" mode since we have upgraded to SP1.
We use some external libraries (we don't have the source code) that were compiled with the original VS2005.
The following manifest is therefore generated:
The application runs fine when the CRT in present in WinSxS (the Policy does the correct redirection). However, if you want to use the "applocal" mode for this particular application.
When running "app.exe" if you get this message: "This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.
Choose external manifest
_USE_RTM_VERSION is newly introduced in SP1 (and previously also in hotfix 919280). It is found in crtassem.h and crtdefs.h. If you use _USE_RTM_VERSION - it must be done in all EXEs and DLLs the app uses. The _USE_RTM_VERSION define determines which CRT version is placed in your manifest file. If it is defined, then the old version number is put in.
If not defined, then it puts the new version in (what ever version ends up shipping with SP1)And no, using _USE_RTM_VERSION does not avoid the runtime rebinding to the new version. The fact is, it only gives the apps the ability to use the originals, but if the new ones are there on the machine in WinSxS, then it uses the new ones instead. Without _USE_RTM_VERSION, if the originals are there and the new ones aren't, then the app doesn't even run.Without using WinSxS, even if you install applocal and if same or newer versions exist in WinSxS, then those in WinSxS will be used instead. However, there is one way to trick the system into never using the new ones from WinSxS, and only using your applocal (original) ones.
For more info see http://blog.kalmbachnet.de/?postid=80
I know of no way to force an app to use a specific version out of WinSxS, if a redirection policy has been installed.
Monday, June 04, 2007
Adobe Apollo
Apollo is the code name for a cross-operating system runtime being developed by Adobe that allows developers to leverage their existing web development skills (Flash, Flex, HTML, JavaScript, Ajax) to build and deploy rich Internet applications (RIAs) to the desktop.
Apollo enables developers to create applications that combine the benefits of web applications – network and user connectivity, rich media content, ease of development, and broad reach – with the strengths of desktop applications – application interactions, local resource access, personal settings, powerful functionality, and rich interactive experiences.
http://labs.adobe.com/technologies/apollo/
Apollo enables developers to create applications that combine the benefits of web applications – network and user connectivity, rich media content, ease of development, and broad reach – with the strengths of desktop applications – application interactions, local resource access, personal settings, powerful functionality, and rich interactive experiences.
http://labs.adobe.com/technologies/apollo/
Darwing Streaming Server basics
Streaming Server Setup
Introduction
The Streaming Server comes in two flavors: Darwin Streaming Server is an Open Source project from Apple. Binaries of Darwin Streaming Server are available for Mac OS X, Red Hat Linux 9, Solaris 9 and Windows 2000/2003/XP. Apple includes QuickTime Streaming Server, a commercially supported version of the server, with Mac OS X Server. QuickTime Streaming Server adds:
QTSS Publisher, content management for streaming media.
QuickTime Broadcaster, a live encoder for live streaming from video/audio sources.
Server Admin, the graphical administration tool for managing Mac OS X Server including the Streaming Server, File Services, Web Services, Mail Services, Directory Services, etc.
QuickTime Streaming Server is formally supported by Apple, while support for the Open Source version is informally provided via mailing lists and on-line resources.
Support and Documentation
Documentation on QuickTime Streaming Server in pdf format is available here (most of the documentation applies to Darwin as well).
A QuickTime/Darwin Streaming Server technology brief in pdf format is available here. Developer documentation is available here. Apple has also posted a good summary of the use and advantages streaming versus http fast-start.
Mailing list support for Darwin Streaming Server and QuickTime Streaming Server is provided informally via the streaming server mailing lists (the users list or the developers list).
Online support resources include the QuickTime section of Apple's support site, and the Streaming Server FAQ.
Download and Install
If you are using Mac OS X Server, QuickTime Streaming Server is already installed. The procedures outlined on this page use the web administration front end for QuickTime Streaming Server. To use web administration you must enable this capability using the Server Admin application in Panther Server. Select the QuickTime Streaming Service and navigate to Access-Settings. Check off "Enable web-based administration", set the password for web administration and save your settings.
If you want to use Darwin Streaming Server, it can be downloaded here. Windows does not include Perl which is required for web administration of Darwin Streaming Server. You can download Active State Perl for Windows here. Make sure to have Perl installed before running the installer on Windows.
Once the Darwin binaries are downloaded, unpack the archive and review the Read Me file, then start the server. On Mac OS X, the server will automatically start after installation. The best way to start the server on Solaris, Linux and Windows is to start up the Perl script that provides web administration:
On Solaris and Linux: /usr/local/sbin/streamingadminserver.pl
On Windows: C:\Perl\bin\perl "C:\Program Files\Darwin Streaming Server\streamingadminserver.pl"
Note: On Windows it may be necessary to reset the administrative password before running streamingserveradmin.pl, the Perl administrative server.
You can create/reset the administrative password on Windows with the following command: C:\Perl\bin\perl "C:\Program Files\Darwin Streaming Server\WinPasswdAssistant.pl"
Once the Perl process is running you can access the web administrative interface by opening your web browser on the server and opening the URL:
http://127.0.0.1:1220
The Streaming Server can reflect live Icecast/Shoutcast broadcasts if you configure a MP3 Broadcast password.
SSL certificate. The server can be administered over SSL if a valid SSL certificate installed. Usually this feature is not enabled during installation.
Media Folder.
This is the "document root" for the streaming server. Media files will be placed in this folder or subdirectories of this folder.
Port 80 Streaming.
The server can be configured to stream over Port 80. If you are not running a web server on port 80 on the same system, it is a good idea to enable this option. This will allow clients to connect to your server through most firewalls.
Once you have navigated through the setup screens, you should see the main window for web server administration on your server. The options configured during setup can be changed in the general settings and port settings panes of the administrative web interface.
Note: You can set up the streamingserveradmin.pl to automatically execute when your server is started if you always want to have the server running (i.e. by editing rc.local on Linux). On Windows 2000/2003 Server the installer adds Darwin Streaming Server as a service in the Service Manager.
Test the Server
To test your server, go to another system on your network with QuickTime installed. Open QuickTime Player and select "Open URL..." from the File Menu:
Enter the rtsp url to your server using it's IP address and the filename sample_100kbit.mov (a sample movie installed in the Movies directory on the server):
The player should open up a stream from your server. Compressed and hinted movies can now be placed in the Movies directory and streamed.
Introduction
The Streaming Server comes in two flavors: Darwin Streaming Server is an Open Source project from Apple. Binaries of Darwin Streaming Server are available for Mac OS X, Red Hat Linux 9, Solaris 9 and Windows 2000/2003/XP. Apple includes QuickTime Streaming Server, a commercially supported version of the server, with Mac OS X Server. QuickTime Streaming Server adds:
QTSS Publisher, content management for streaming media.
QuickTime Broadcaster, a live encoder for live streaming from video/audio sources.
Server Admin, the graphical administration tool for managing Mac OS X Server including the Streaming Server, File Services, Web Services, Mail Services, Directory Services, etc.
QuickTime Streaming Server is formally supported by Apple, while support for the Open Source version is informally provided via mailing lists and on-line resources.
Support and Documentation
Documentation on QuickTime Streaming Server in pdf format is available here (most of the documentation applies to Darwin as well).
A QuickTime/Darwin Streaming Server technology brief in pdf format is available here. Developer documentation is available here. Apple has also posted a good summary of the use and advantages streaming versus http fast-start.
Mailing list support for Darwin Streaming Server and QuickTime Streaming Server is provided informally via the streaming server mailing lists (the users list or the developers list).
Online support resources include the QuickTime section of Apple's support site, and the Streaming Server FAQ.
Download and Install
If you are using Mac OS X Server, QuickTime Streaming Server is already installed. The procedures outlined on this page use the web administration front end for QuickTime Streaming Server. To use web administration you must enable this capability using the Server Admin application in Panther Server. Select the QuickTime Streaming Service and navigate to Access-Settings. Check off "Enable web-based administration", set the password for web administration and save your settings.
If you want to use Darwin Streaming Server, it can be downloaded here. Windows does not include Perl which is required for web administration of Darwin Streaming Server. You can download Active State Perl for Windows here. Make sure to have Perl installed before running the installer on Windows.
Once the Darwin binaries are downloaded, unpack the archive and review the Read Me file, then start the server. On Mac OS X, the server will automatically start after installation. The best way to start the server on Solaris, Linux and Windows is to start up the Perl script that provides web administration:
On Solaris and Linux: /usr/local/sbin/streamingadminserver.pl
On Windows: C:\Perl\bin\perl "C:\Program Files\Darwin Streaming Server\streamingadminserver.pl"
Note: On Windows it may be necessary to reset the administrative password before running streamingserveradmin.pl, the Perl administrative server.
You can create/reset the administrative password on Windows with the following command: C:\Perl\bin\perl "C:\Program Files\Darwin Streaming Server\WinPasswdAssistant.pl"
Once the Perl process is running you can access the web administrative interface by opening your web browser on the server and opening the URL:
http://127.0.0.1:1220
The Streaming Server can reflect live Icecast/Shoutcast broadcasts if you configure a MP3 Broadcast password.
SSL certificate. The server can be administered over SSL if a valid SSL certificate installed. Usually this feature is not enabled during installation.
Media Folder.
This is the "document root" for the streaming server. Media files will be placed in this folder or subdirectories of this folder.
Port 80 Streaming.
The server can be configured to stream over Port 80. If you are not running a web server on port 80 on the same system, it is a good idea to enable this option. This will allow clients to connect to your server through most firewalls.
Once you have navigated through the setup screens, you should see the main window for web server administration on your server. The options configured during setup can be changed in the general settings and port settings panes of the administrative web interface.
Note: You can set up the streamingserveradmin.pl to automatically execute when your server is started if you always want to have the server running (i.e. by editing rc.local on Linux). On Windows 2000/2003 Server the installer adds Darwin Streaming Server as a service in the Service Manager.
Test the Server
To test your server, go to another system on your network with QuickTime installed. Open QuickTime Player and select "Open URL..." from the File Menu:
Enter the rtsp url to your server using it's IP address and the filename sample_100kbit.mov (a sample movie installed in the Movies directory on the server):
The player should open up a stream from your server. Compressed and hinted movies can now be placed in the Movies directory and streamed.
Thursday, May 31, 2007
Microsoft Surface
Who say that apple is the best company for user interface.
Look at how microsoft is giving a lesson to apple
http://www.microsoft.com/surface/
Impressive ! Worth the virtual tour.
Seams to me be the next operating system !
Microsoft CEO Steve Ballmer showed off Surface, novel interface technology that powers a touch-screen tabletop on which you can create and manipulate images and content with gestures of your hands. The initial units will sell for between $5,000 and $10,000 each and are likely to find homes in hotels, casinos and retail stores (along with the dens of wealthy geeks). Consensus of onlookers: Pretty cool.
Look at how microsoft is giving a lesson to apple
http://www.microsoft.com/surface/
Impressive ! Worth the virtual tour.
Seams to me be the next operating system !
Microsoft CEO Steve Ballmer showed off Surface, novel interface technology that powers a touch-screen tabletop on which you can create and manipulate images and content with gestures of your hands. The initial units will sell for between $5,000 and $10,000 each and are likely to find homes in hotels, casinos and retail stores (along with the dens of wealthy geeks). Consensus of onlookers: Pretty cool.
Wednesday, May 16, 2007
Unix Thread Versus Windows Thread
A good article for people who are trying to build efficient cross platform threading class
to be build on Linux or Windows
http://www.microsoft.com/technet/interopMigration/taskstools/migrate/unix/ucamg/ch03uav3.mspx
to be build on Linux or Windows
http://www.microsoft.com/technet/interopMigration/taskstools/migrate/unix/ucamg/ch03uav3.mspx
Tuesday, May 15, 2007
Visual Studio Project to Makefile for cross platform project
Visual Studio 6
To export a makefile
From the Project menu, select Export Makefile.
In the Export Makefile(s) dialog box, select the project(s) for which you want to create a .mak file.
A separate .mak file is created for each project you select.
If you want Visual C++ to update dependency information, check the Write dependencies when writing makefiles checkbox.
If you know that you want to automatically export the .mak file each time you update this project, use the following procedure.
To automatically export a makefile when the project is updated
1 From the Tools menu, choose Options.
2 On the Build tab, check the Export makefile when saving project file checkbox.
Selecting this option increases the time it takes to save the project.
3 To automatically write dependency information every time you export a makefile
From the Tools menu, choose Options.
4 On the Build tab, check the Always write dependencies when writing makefiles checkbox.
Selecting this option considerably increases the time it takes to write the .mak file.
5 To specify per-configuration dependency information
From the Project menu, choose Settings.
On the General tab, check the Allow per-configuration dependencies checkbox.
Selecting this option increases the time it takes to write the dependency information, according to the number of build configurations in the project.
http://msdn2.microsoft.com/en-us/library/aa233950(VS.60).aspx
Visual Studio .NET
Comvert the project to Visual Studio 6 using
stephane rodriguez prjconverter then export to makefile using Visual Studio 6
http://www.codeproject.com/tools/prjconverter.asp?df=100&forumid=10069&exp=0&select=1288997
WineMaker
The Winelib development toolkit
http://www.winehq.org/site/docs/winelib-guide/winelib-toolkit
Other
would be nice to implement a plugin for Visual Studio 2005.
I am using one to generate doxygen automatically. Merging code from winemaker
into a VS2005 plugin would be a great idea.
What about exporting the project towards those fancy building script as well
SConstruct
ANT
etc...
To export a makefile
From the Project menu, select Export Makefile.
In the Export Makefile(s) dialog box, select the project(s) for which you want to create a .mak file.
A separate .mak file is created for each project you select.
If you want Visual C++ to update dependency information, check the Write dependencies when writing makefiles checkbox.
If you know that you want to automatically export the .mak file each time you update this project, use the following procedure.
To automatically export a makefile when the project is updated
1 From the Tools menu, choose Options.
2 On the Build tab, check the Export makefile when saving project file checkbox.
Selecting this option increases the time it takes to save the project.
3 To automatically write dependency information every time you export a makefile
From the Tools menu, choose Options.
4 On the Build tab, check the Always write dependencies when writing makefiles checkbox.
Selecting this option considerably increases the time it takes to write the .mak file.
5 To specify per-configuration dependency information
From the Project menu, choose Settings.
On the General tab, check the Allow per-configuration dependencies checkbox.
Selecting this option increases the time it takes to write the dependency information, according to the number of build configurations in the project.
http://msdn2.microsoft.com/en-us/library/aa233950(VS.60).aspx
Visual Studio .NET
Comvert the project to Visual Studio 6 using
stephane rodriguez prjconverter then export to makefile using Visual Studio 6
http://www.codeproject.com/tools/prjconverter.asp?df=100&forumid=10069&exp=0&select=1288997
WineMaker
The Winelib development toolkit
http://www.winehq.org/site/docs/winelib-guide/winelib-toolkit
Other
would be nice to implement a plugin for Visual Studio 2005.
I am using one to generate doxygen automatically. Merging code from winemaker
into a VS2005 plugin would be a great idea.
What about exporting the project towards those fancy building script as well
SConstruct
ANT
etc...
Monday, May 07, 2007
Using mouse and clipboard between two separate Operating Systems running on two separate boxes
Have you ever wanted to be able to cut and past text between two separate Operating System running on two diffrence box. Have you ever wanted to use one mouse on multiple Operating System.
What you are looking for is Synergy.
Synergy lets you easily share a single mouse and keyboard between multiple computers with different operating systems, each with its own display, without special hardware. It's intended for users with multiple computers on their desk since each system uses its own monitor(s).
Redirecting the mouse and keyboard is as simple as moving the mouse off the edge of your screen. Synergy also merges the clipboards of all the systems into one, allowing cut-and-paste between systems. Furthermore, it synchronizes screen savers so they all start and stop together and, if screen locking is enabled, only one screen requires a password to unlock them all. Learn more about how it works.
http://synergy2.sourceforge.net/
What you are looking for is Synergy.
Synergy lets you easily share a single mouse and keyboard between multiple computers with different operating systems, each with its own display, without special hardware. It's intended for users with multiple computers on their desk since each system uses its own monitor(s).
Redirecting the mouse and keyboard is as simple as moving the mouse off the edge of your screen. Synergy also merges the clipboards of all the systems into one, allowing cut-and-paste between systems. Furthermore, it synchronizes screen savers so they all start and stop together and, if screen locking is enabled, only one screen requires a password to unlock them all. Learn more about how it works.
http://synergy2.sourceforge.net/
Tuesday, May 01, 2007
Show TIB Under The Hood From Matt Pietrek
Matt Pietrek is the author of Windows 95 System Programming Secrets (IDG Books, 1995).
http://www.microsoft.com/msj/archive/s2ce.aspx
While Windows NT™ and Windows® 95 are quite different under the hood, they both share a key system data structure that many programmers aren't aware of. To be a bit more precise, certain fields of this data structure are shared. Regardless of the differences, this structure is used extensively, and is even accessed by compiler-generated code. That's right, your C++ compiler generates code to access system level information directly.
What structure am I talking about? It goes by at least two different names. The Windows 95 code calls it a Thread Information Block (TIB).
In Windows NT, it's called the Thread Environment Block (TEB). However, I've seen it referred to as a TIB in some Windows NT header files so, for the purposes of this column, I'll refer to it as a TIB.
What's in a Thread Information Block that makes it so special? As its name implies, the data found in a TIB relates to threads, and there's a TIB for each thread in the system.
In fields shared by Windows NT and Windows 95, you'll find information like a pointer to the thread's structured exception handler list, the location of the thread's stack, and the location of the thread local storage slots. Other fields in the TIB differ between Windows NT and Windows 95.
You might be surprised to learn that the TIB didn't first appear in Windows NT or Windows 95. The TIB has its ancestry in OS/2, before Microsoft created Windows NT, and it still exists in OS/2 today. In fact, the OS/2 TIB shares much of the same format, and is accessed in the same manner as under Win32®. There's even a line in a Microsoft header file (NTDDK.H) that says <<>>
// This structure MUST MATCH OS/2 V2.0!
<<>>
How would you get at the TIB if you were inclined to go poking around? It's as easy as looking at what the FS register points to. Hey! Didn't segments and segment registers go away in Win32? For the most part, that's true. However, in all Intel-based Win32 implementations (even the forgotten stepchild, Win32s), the FS register points to the TIB. Thus, all of the structure offsets that I'll detail later can be used as offsets into the segment pointed to by the FS register. For example, FS:[0] points to the structured exception handling chain, while FS:[2C] points to the thread's thread local storage array.
I just mentioned that compilers access the TIB structure directly. Let's look at a small example to see this in action. This is a very small C program that uses structured exception handling:
int main()
{
__try
{ int i = 0;
}
__except( 1 )
{ int j = 1: }
return 0;
}
When compiled, the first part of the resulting code looks like this: 401000: PUSH EBP
401001: MOV EBP,ESP
401003: PUSH FF
401005: PUSH 00404000
40100A: PUSH 00401140
40100F: MOV EAX,FS:[00000000]
401015: PUSH EAX
401016: MOV DWORD PTR FS:[00000000],ESP
The thing to notice is the series of PUSH instructions. They create a data structure on the stack (sort of like an invisible local variable). The instruction at offset 0x_40100F retrieves the head of the structured exception handling chain out of the thread information block and stores it into EAX-this is where the FS:[00000000] part of the instruction comes from. The code then pushes the current head of the structured exception handling chain list onto the stack. This finishes the process of creating a local data structure on the stack. Finally, the MOV DWORD PTR FS:[00000000],ESP instruction changes the head of the structured exception handling chain to point at the newly created data structure.
The key point is that Win32 compilers implicitly know about the TIB and generate code that accesses it. Because the compiler can't know which Win32 system the code will run on, you can safely assume that any compiler-generated code that references the FS segment uses fields common between Win32 platforms.
Common Fields in the TIB You just saw one example of a TIB structure field that's common to all Win32 implementations. In this section, I'll list all of the common fields, along with a short description. The fields that differ between Win32 implementations are described later.
There are several different header files floating around that define the fields of a TIB. Unfortunately, they're not always consistent with each other or complete. In the Windows NT DDK, you'll find a structure called an NT_TIB defined in NTDDK.H.
In the Windows NT 3.51 service pack 3 SDK update, a new WINNT.H was added that also defines an NT_TIB structure.
In addition, someone from the Windows 95 team posted online a snippet from an .H file that described a TIB.
In my descriptions, I've tried to use the most descriptive name. You'll see these names in the TIB.H file included with the SHOWTIB program I'll present later on.
The 00h DWORD pvExcept field contains a pointer to the head of the thread's structured exception handling chain. The chain is a linked list of EXCEPTION_REGISTRATION_RECORD structures (which unfortunately are not defined in any official .H file).
For more information on the structured exception handling chain, you might refer to chapter 3 of my book "Windows 95 System Programming Secrets."
The 04h DWORD pvStackUserTop field contains the linear address of the topmost address of the thread's stack. Put another way, at no point will this thread have a stack pointer value that's greater than or equal to the value of this field.
The 08h DWORD pvStackUserBase field contains the linear address of the lowest committed page in the thread's user mode stack. As the thread uses successively lower addresses in the stack, those pages will be committed, and this field will be updated accordingly.
The 14h DWORD pvArbitrary field is theoretically available for applications to use however they want. It's almost like an extra thread local storage slot for you to use, although I've never seen an application use it.
The 18h DWORD ptibSelf field holds the linear address of the TIB. Put another way, the TIB block contains a pointer to itself. Why bother doing this? If the TIB's fields are used extensively, it makes sense for 32-bit code to read and write the TIB using regular pointers rather than using a segment register override (for example, by using FS:[xxxxxxxx]). The SHOWTIB program presented later uses this field.
The 2Ch DWORD pvTLSArray field contains a pointer to the thread local storage (TLS) slots for the thread. For example, if you had a TLS index value of 4, you could take this pointer, add 10h to it ( 4 * sizeof(DWORD) ), and retrieve the TLS value directly. Knowing that this field points to the thread's TLS slots, you could write your own versions of TlsSetValue and TlsGetValue easily. If you use _ _declspec(thread) variables in your code, check out the ASM code emitted by your compiler.
You'll find that it uses this field.
The location of the TLS slots is quite different between Windows NT and Windows 95. In Windows NT, this field contains a null pointer until the first time a TLS slot is used in the thread, then the system allocates memory for the TLS slots out of the default process heap. (In Windows NT, a buffer overrun of a HeapAlloc'ed block could trash your TLS data.)
Under Windows 95, this field always points to the TLS slots that are kept as part of the Ring 3 thread database.
Windows NT TIB fields
The meaning of some TIB data differs depending on whether you're running under Windows NT or Windows 95. This section of the OS/2 subsystem-Windows NT support running OS/2 1.X applications. For regular Win32 apps, this field appears to always be zero.
The 10h DWORD FiberData field's meaning depends on what version of Windows NT the thread is running. In the Windows NT 3.51 service pack 3 SDK update, WINNT.H describes this field as pointing to fiber data. Fibers are described in the accompanying HLP file as lightweight threads that are scheduled manually. Prior to the service pack update, this field was named "Version". Presumably this means what version of the system the thread expects to be running on, but I was unable to make sense of the values in this field.
The 20h DWORD processID field holds the process ID of the thread. The GetCurrentProcessId function in Windows NT 3.51 simply returns whatever is in this field.
The 24h DWORD threadID field holds the thread's ID. The GetCurrentThreadId function in Windows NT 3.51 returns the value in this field.
The segment pointed at by the Windows NT TIB actually extends far beyond the fields that I've described here. I've only mentioned the fields that fall within the first 34h bytes (the size of a Windows 95 TIB).
Windows 95 TIB fields
While the Windows NT TIB fields are relatively sedate, the Windows 95 TIB contains a fair amount of intriguing information. This section covers fields specific to the Windows 95 TIB.
The 0CH WORD pvTDB contains the task database selector for the task associated with the thread. The task database is a segment allocated from the 16-bit global heap, and the handle is known as an HTASK.
In Windows 95, every process (even a Win32 process) has a task database created for it. The 0EH WORD pvThunkSS field contains the selector that Windows 95 uses as the 16-bit stack selector when a thread thunks from 32-bit code to 16-bit code.
The 1CH WORD TIBFlags field is intended to hold various bit flags. The only known value is TIB_WIN32 (that is, 1).
If the low bit of this value is set, it's a 32-bit thread, otherwise it's a thread from a 16-bit process. The 1Eh WORD Win16MutexCount field is related to the thread's ownership of the Win16Mutex, which is a global critical section that only allows one thread at a time to be in 16-bit code.
Normally, this field's value is -1, which indicates that the thread doesn't own the Win16Mutex. As the thread enters and leaves thinking code, the value of this field is incremented and decremented accordingly.
The 20h DWORD DebugContext field normally contains the value zero. However, when you're debugging the thread's process, this field contains a pointer to a structure that contains register values and is similar to, but not the same as, the CONTEXT structure defined in WINNT.H.
The 24h DWORD pCurrentPriority field points to a DWORD containing the thread's scheduling priority. This will be some value between zero (lowest) and 31 (highest). The DWORD pointed to by this field is above 3GB in linear memory, which places it in VxD land. This makes sense, as threads are scheduled by the ring 0 Virtual Machine Manager (VMM). For normal priority threads, the priority DWORD will contain 9.
The 28h DWORD pvQueue field contains the message queue selector assigned to the thread.
Message queues are the means by which window messages get to the appropriate windows. In Windows 95, each thread can have its own message queue, but it is initially created without one. Therefore, this field may contain zero.
The 30h PVOID* pProcess field contains a linear address for the process database representing the process that owns the thread. However, this is not the same as a process handle or process ID.
Some Random Notes on TIBs
As I was experimenting with TIBs for this column, I came across some tidbits of information worth passing on.
First, in Windows 95, at offset 52h in each task database segment, you'll find the TIB selector for the primary thread in the process. At offset 54h in the task database, you'll find the linear address of the TIB. This is particularly interesting in that task databases are used by the 16-bit components of Windows 95. It appears that the 16-bit components may occasionally access thread-specific data in the TIB.
I also noticed the different uses of the FS register between Windows NT and Windows 95. Under Windows NT, the FS register is always the same for each thread's TIB. This implies that the linear address for the FS selector has to change whenever a thread switch occurs. In contrast, Windows 95 dedicates a different selector for each TIB (and hence, for each thread). The linear address of a Windows 95 TIB selector doesn't change.
I'll let you guess which method is kinder to system resources.
The SHOWTIB program
To bring the TIB to life, I wrote the SHOWTIB program (see Figure 1). SHOWTIB is a simple command-line program with two goals. The first is to create one or more threads. (You specify the actual number of threads on the command line. For example, "SHOWTIB 5" tells SHOWTIB to spin off five threads and display their TIBs.) The second goal is to display the various fields of each thread's TIB structure once all the threads are running. I show only the TIB structures for threads created by the primary thread, not for the primary thread itself.
In displaying the TIB for each thread, SHOWTIB first displays the fields common to all Win32 operating systems, then decides whether it's running on Windows NT or Windows 95. Depending on which system is running, SHOWTIB displays specific fields in the TIB relevant to the operating system. To make each TIB display come out coherent and in one piece, the DisplayTIB function guards the display code with a critical section.
There are two interesting pieces of code in SHOWTIB.CPP. Near the start of the DisplayTIB function, the code uses a bit of in-line assembler to grab the field at offset 18h in the TIB and stash it away into a pointer. Offset 18h is the linear address of the TIB. I did this so I could access the rest of the TIB with a regular pointer.
The alternative would have meant using in-line assembler and FS segment overrides to retrieve all of the values. Win32 compilers simply don't have a way to let you easily read from any segment other than the data segment (the DS register).
The second interesting piece of code is near the end of main.
After creating all the threads and storing all the corresponding thread handles into an array, I call WaitForMultipleObjects, passing in the array of thread handles. If I didn't do this, the primary thread could return from the main routine and call the exit function before the worker-bee threads had terminated. The result would be an incomplete display of all the various TIBs.
While Windows NT and Windows 95 are quite different under the hood, the TIB is one of the few areas you can rely on to be the same.
This isn't a coincidence; since threading is such an integral part of both operating systems, it's only natural that some common method of supplying thread-specific information would be needed.
The TIB is not described in any official documentation other than .H files. Nonetheless, it's an integral part of the Win32 specification that all Win32 implementations must conform to.
http://www.microsoft.com/msj/archive/s2ce.aspx
While Windows NT™ and Windows® 95 are quite different under the hood, they both share a key system data structure that many programmers aren't aware of. To be a bit more precise, certain fields of this data structure are shared. Regardless of the differences, this structure is used extensively, and is even accessed by compiler-generated code. That's right, your C++ compiler generates code to access system level information directly.
What structure am I talking about? It goes by at least two different names. The Windows 95 code calls it a Thread Information Block (TIB).
In Windows NT, it's called the Thread Environment Block (TEB). However, I've seen it referred to as a TIB in some Windows NT header files so, for the purposes of this column, I'll refer to it as a TIB.
What's in a Thread Information Block that makes it so special? As its name implies, the data found in a TIB relates to threads, and there's a TIB for each thread in the system.
In fields shared by Windows NT and Windows 95, you'll find information like a pointer to the thread's structured exception handler list, the location of the thread's stack, and the location of the thread local storage slots. Other fields in the TIB differ between Windows NT and Windows 95.
You might be surprised to learn that the TIB didn't first appear in Windows NT or Windows 95. The TIB has its ancestry in OS/2, before Microsoft created Windows NT, and it still exists in OS/2 today. In fact, the OS/2 TIB shares much of the same format, and is accessed in the same manner as under Win32®. There's even a line in a Microsoft header file (NTDDK.H) that says <<>>
// This structure MUST MATCH OS/2 V2.0!
<<>>
How would you get at the TIB if you were inclined to go poking around? It's as easy as looking at what the FS register points to. Hey! Didn't segments and segment registers go away in Win32? For the most part, that's true. However, in all Intel-based Win32 implementations (even the forgotten stepchild, Win32s), the FS register points to the TIB. Thus, all of the structure offsets that I'll detail later can be used as offsets into the segment pointed to by the FS register. For example, FS:[0] points to the structured exception handling chain, while FS:[2C] points to the thread's thread local storage array.
I just mentioned that compilers access the TIB structure directly. Let's look at a small example to see this in action. This is a very small C program that uses structured exception handling:
int main()
{
__try
{ int i = 0;
}
__except( 1 )
{ int j = 1: }
return 0;
}
When compiled, the first part of the resulting code looks like this: 401000: PUSH EBP
401001: MOV EBP,ESP
401003: PUSH FF
401005: PUSH 00404000
40100A: PUSH 00401140
40100F: MOV EAX,FS:[00000000]
401015: PUSH EAX
401016: MOV DWORD PTR FS:[00000000],ESP
The thing to notice is the series of PUSH instructions. They create a data structure on the stack (sort of like an invisible local variable). The instruction at offset 0x_40100F retrieves the head of the structured exception handling chain out of the thread information block and stores it into EAX-this is where the FS:[00000000] part of the instruction comes from. The code then pushes the current head of the structured exception handling chain list onto the stack. This finishes the process of creating a local data structure on the stack. Finally, the MOV DWORD PTR FS:[00000000],ESP instruction changes the head of the structured exception handling chain to point at the newly created data structure.
The key point is that Win32 compilers implicitly know about the TIB and generate code that accesses it. Because the compiler can't know which Win32 system the code will run on, you can safely assume that any compiler-generated code that references the FS segment uses fields common between Win32 platforms.
Common Fields in the TIB You just saw one example of a TIB structure field that's common to all Win32 implementations. In this section, I'll list all of the common fields, along with a short description. The fields that differ between Win32 implementations are described later.
There are several different header files floating around that define the fields of a TIB. Unfortunately, they're not always consistent with each other or complete. In the Windows NT DDK, you'll find a structure called an NT_TIB defined in NTDDK.H.
In the Windows NT 3.51 service pack 3 SDK update, a new WINNT.H was added that also defines an NT_TIB structure.
In addition, someone from the Windows 95 team posted online a snippet from an .H file that described a TIB.
In my descriptions, I've tried to use the most descriptive name. You'll see these names in the TIB.H file included with the SHOWTIB program I'll present later on.
The 00h DWORD pvExcept field contains a pointer to the head of the thread's structured exception handling chain. The chain is a linked list of EXCEPTION_REGISTRATION_RECORD structures (which unfortunately are not defined in any official .H file).
For more information on the structured exception handling chain, you might refer to chapter 3 of my book "Windows 95 System Programming Secrets."
The 04h DWORD pvStackUserTop field contains the linear address of the topmost address of the thread's stack. Put another way, at no point will this thread have a stack pointer value that's greater than or equal to the value of this field.
The 08h DWORD pvStackUserBase field contains the linear address of the lowest committed page in the thread's user mode stack. As the thread uses successively lower addresses in the stack, those pages will be committed, and this field will be updated accordingly.
The 14h DWORD pvArbitrary field is theoretically available for applications to use however they want. It's almost like an extra thread local storage slot for you to use, although I've never seen an application use it.
The 18h DWORD ptibSelf field holds the linear address of the TIB. Put another way, the TIB block contains a pointer to itself. Why bother doing this? If the TIB's fields are used extensively, it makes sense for 32-bit code to read and write the TIB using regular pointers rather than using a segment register override (for example, by using FS:[xxxxxxxx]). The SHOWTIB program presented later uses this field.
The 2Ch DWORD pvTLSArray field contains a pointer to the thread local storage (TLS) slots for the thread. For example, if you had a TLS index value of 4, you could take this pointer, add 10h to it ( 4 * sizeof(DWORD) ), and retrieve the TLS value directly. Knowing that this field points to the thread's TLS slots, you could write your own versions of TlsSetValue and TlsGetValue easily. If you use _ _declspec(thread) variables in your code, check out the ASM code emitted by your compiler.
You'll find that it uses this field.
The location of the TLS slots is quite different between Windows NT and Windows 95. In Windows NT, this field contains a null pointer until the first time a TLS slot is used in the thread, then the system allocates memory for the TLS slots out of the default process heap. (In Windows NT, a buffer overrun of a HeapAlloc'ed block could trash your TLS data.)
Under Windows 95, this field always points to the TLS slots that are kept as part of the Ring 3 thread database.
Windows NT TIB fields
The meaning of some TIB data differs depending on whether you're running under Windows NT or Windows 95. This section of the OS/2 subsystem-Windows NT support running OS/2 1.X applications. For regular Win32 apps, this field appears to always be zero.
The 10h DWORD FiberData field's meaning depends on what version of Windows NT the thread is running. In the Windows NT 3.51 service pack 3 SDK update, WINNT.H describes this field as pointing to fiber data. Fibers are described in the accompanying HLP file as lightweight threads that are scheduled manually. Prior to the service pack update, this field was named "Version". Presumably this means what version of the system the thread expects to be running on, but I was unable to make sense of the values in this field.
The 20h DWORD processID field holds the process ID of the thread. The GetCurrentProcessId function in Windows NT 3.51 simply returns whatever is in this field.
The 24h DWORD threadID field holds the thread's ID. The GetCurrentThreadId function in Windows NT 3.51 returns the value in this field.
The segment pointed at by the Windows NT TIB actually extends far beyond the fields that I've described here. I've only mentioned the fields that fall within the first 34h bytes (the size of a Windows 95 TIB).
Windows 95 TIB fields
While the Windows NT TIB fields are relatively sedate, the Windows 95 TIB contains a fair amount of intriguing information. This section covers fields specific to the Windows 95 TIB.
The 0CH WORD pvTDB contains the task database selector for the task associated with the thread. The task database is a segment allocated from the 16-bit global heap, and the handle is known as an HTASK.
In Windows 95, every process (even a Win32 process) has a task database created for it. The 0EH WORD pvThunkSS field contains the selector that Windows 95 uses as the 16-bit stack selector when a thread thunks from 32-bit code to 16-bit code.
The 1CH WORD TIBFlags field is intended to hold various bit flags. The only known value is TIB_WIN32 (that is, 1).
If the low bit of this value is set, it's a 32-bit thread, otherwise it's a thread from a 16-bit process. The 1Eh WORD Win16MutexCount field is related to the thread's ownership of the Win16Mutex, which is a global critical section that only allows one thread at a time to be in 16-bit code.
Normally, this field's value is -1, which indicates that the thread doesn't own the Win16Mutex. As the thread enters and leaves thinking code, the value of this field is incremented and decremented accordingly.
The 20h DWORD DebugContext field normally contains the value zero. However, when you're debugging the thread's process, this field contains a pointer to a structure that contains register values and is similar to, but not the same as, the CONTEXT structure defined in WINNT.H.
The 24h DWORD pCurrentPriority field points to a DWORD containing the thread's scheduling priority. This will be some value between zero (lowest) and 31 (highest). The DWORD pointed to by this field is above 3GB in linear memory, which places it in VxD land. This makes sense, as threads are scheduled by the ring 0 Virtual Machine Manager (VMM). For normal priority threads, the priority DWORD will contain 9.
The 28h DWORD pvQueue field contains the message queue selector assigned to the thread.
Message queues are the means by which window messages get to the appropriate windows. In Windows 95, each thread can have its own message queue, but it is initially created without one. Therefore, this field may contain zero.
The 30h PVOID* pProcess field contains a linear address for the process database representing the process that owns the thread. However, this is not the same as a process handle or process ID.
Some Random Notes on TIBs
As I was experimenting with TIBs for this column, I came across some tidbits of information worth passing on.
First, in Windows 95, at offset 52h in each task database segment, you'll find the TIB selector for the primary thread in the process. At offset 54h in the task database, you'll find the linear address of the TIB. This is particularly interesting in that task databases are used by the 16-bit components of Windows 95. It appears that the 16-bit components may occasionally access thread-specific data in the TIB.
I also noticed the different uses of the FS register between Windows NT and Windows 95. Under Windows NT, the FS register is always the same for each thread's TIB. This implies that the linear address for the FS selector has to change whenever a thread switch occurs. In contrast, Windows 95 dedicates a different selector for each TIB (and hence, for each thread). The linear address of a Windows 95 TIB selector doesn't change.
I'll let you guess which method is kinder to system resources.
The SHOWTIB program
To bring the TIB to life, I wrote the SHOWTIB program (see Figure 1). SHOWTIB is a simple command-line program with two goals. The first is to create one or more threads. (You specify the actual number of threads on the command line. For example, "SHOWTIB 5" tells SHOWTIB to spin off five threads and display their TIBs.) The second goal is to display the various fields of each thread's TIB structure once all the threads are running. I show only the TIB structures for threads created by the primary thread, not for the primary thread itself.
In displaying the TIB for each thread, SHOWTIB first displays the fields common to all Win32 operating systems, then decides whether it's running on Windows NT or Windows 95. Depending on which system is running, SHOWTIB displays specific fields in the TIB relevant to the operating system. To make each TIB display come out coherent and in one piece, the DisplayTIB function guards the display code with a critical section.
There are two interesting pieces of code in SHOWTIB.CPP. Near the start of the DisplayTIB function, the code uses a bit of in-line assembler to grab the field at offset 18h in the TIB and stash it away into a pointer. Offset 18h is the linear address of the TIB. I did this so I could access the rest of the TIB with a regular pointer.
The alternative would have meant using in-line assembler and FS segment overrides to retrieve all of the values. Win32 compilers simply don't have a way to let you easily read from any segment other than the data segment (the DS register).
The second interesting piece of code is near the end of main.
After creating all the threads and storing all the corresponding thread handles into an array, I call WaitForMultipleObjects, passing in the array of thread handles. If I didn't do this, the primary thread could return from the main routine and call the exit function before the worker-bee threads had terminated. The result would be an incomplete display of all the various TIBs.
While Windows NT and Windows 95 are quite different under the hood, the TIB is one of the few areas you can rely on to be the same.
This isn't a coincidence; since threading is such an integral part of both operating systems, it's only natural that some common method of supplying thread-specific information would be needed.
The TIB is not described in any official documentation other than .H files. Nonetheless, it's an integral part of the Win32 specification that all Win32 implementations must conform to.
Subscribe to:
Posts (Atom)