Integrating Xgrid into Cocoa Applications, Part 2
Pages: 1, 2, 3, 4, 5
We can now finish off the applyFilters:toFilesWithPaths:forOutputDirectoryPath:
method.
// Copy PIL for the input directory
NSBundle *bundle = [NSBundle bundleForClass:[self class]];
NSString *pilPath = [bundle pathForResource:@"PIL" ofType:nil];
NSAssert( nil != pilPath, @"PIL path was nil." );
[fm copyPath:pilPath toDirectoryAtPath:inputDirPath];
// Add subtask to task
NSString *scriptPath =
[bundle pathForResource:@"agentrunscript" ofType:@"py"];
[task addSubTaskWithIdentifier:[NSNumber numberWithInt:subTaskIndex]
launchPath:scriptPath
workingDirectoryPath:inputDirPath
outputDirectoryPath:taskOutputDirPath
standardInputPath:siPath
standardOutputPath:nil];
}
[task launch];
}
Still inside the loop over subtasks, we copy the directory containing PIL
to the input directory of the subtask. The subtask is then added to the distributed
task, using the method discussed earlier, setting the launch path, input and
output directories of the subtask, and the standard input file. We don't need
the standard output, so nil is passed for that. Finally, when all
subtasks have been added to the distributed task, the task is launched.
The progress of the DistributedTask is monitored using its delegate
methods. The PIController was set as the delegate to the task,
so it can implement the methods, and act upon them as required. We will take
a look at the method called when the DistributedTask has finished
running all its subtasks on Xgrid: distributedTaskDidFinishSubTasks:.
The method first checks that the output directory that the user requested
exists. If not, it creates it. If it does exist, and is not a directory, an
NSAssert ensures an exception is raised. In a more robust app,
you would want to handle this better, by informing the user of the problem.
-(void)distributedTaskDidFinishSubTasks:(DistributedTask *)distributedTask {
NSFileManager *fm = [NSFileManager defaultManager];
// Ensure output directory exists, and that it is a directory.
BOOL isDir;
if ( [fm fileExistsAtPath:[self outputDirectoryPath] isDirectory:&isDir] ) {
NSAssert( isDir, @"Output directory path supplied was not a directory." );
}
else {
[fm createDirectoryAtPath:[self outputDirectoryPath] attributes:nil];
}
Next, the filtered photos, which should be in the output directory of the
DistributedTask, are moved to the user's chosen output destination.
// Move task output files to the output directory
NSString *taskOutputDirPath =
[taskTempDirPath stringByAppendingPathComponent:@"output"];
[fm changeCurrentDirectoryPath:taskOutputDirPath];
NSDirectoryEnumerator *en = [fm enumeratorAtPath:taskOutputDirPath];
NSString *relativePath; // Path relative to taskOutputDirPath
while ( relativePath = [en nextObject] ) {
if ( NSOrderedSame !=
[[relativePath pathExtension] caseInsensitiveCompare:@"jpg"] &&
NSOrderedSame !=
[[relativePath pathExtension] caseInsensitiveCompare:@"jpeg"] )
continue;
NSString *fileOutputPath =
[[self outputDirectoryPath] stringByAppendingPathComponent:
[relativePath lastPathComponent]];
if ( [fm fileExistsAtPath:fileOutputPath] )
[fm removeFileAtPath:fileOutputPath handler:nil];
[fm linkPath:relativePath toPath:fileOutputPath handler:nil];
}
An NSDirectoryEnumerator is employed for this operation. An NSDirectoryEnumerator
traverses the contents of a directory, including subdirectories. For each file
found, we check if it is a JPEG, and, if so, link it to the output directory.
Yes, in this case we do use link, instead of copy, because the operation doesn't
pose any threat to the original photos, and is faster.
Finally, we clean up, by removing the entire task temporary directory, which includes all of the files and directories used by the subtasks.
// Remove temporary directory
[fm removeFileAtPath:taskTempDirPath handler:nil];
...
}
Odds and Ends
We have now covered the parts of Photo Industry that relate directly to Xgrid. If you download the source, you will see there is a lot of other stuff in the application, which could also be useful in your own apps. Photo Industry makes use of drag and drop, for example, and the new Cocoa bindings layer. A good intro to the drag-and-drop techniques can be found on CocoaDevCentral. Bindings are also covered by CocoaDevCentral here, and don't forget that old stalwart Mike Beam, who has recently written on the topic here.
I hope this two-part article has demonstrated the potential of Xgrid in
non-scientific Cocoa applications. We have had to do a lot of work in order
to leverage that potential, using the command-line xgrid tool,
but in all likelihood WWDC will alleviate that in the near future. Hopefully,
this article will be irrelevant after June 28, 2004, when SJ finally unveils
Apple's vision for Xgrid and the future of distributed computation. To be continued
...
Drew McCormack works at the Free University in Amsterdam, and develops the Cocoa shareware Trade Strategist.
Return to the Mac DevCenter
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 5 of 5.
-
xgrid command hanging?
2004-05-26 13:37:36 jamesreynolds [Reply | View]
-
xgrid command hanging?
2004-05-26 22:36:28 drewmccormack [Reply | View]
It wouldn't handle a hanging thread. It would just wait forever I suspect. You could add a timeout for this, I guess.
If you were going to process so many images, you would probably want to split things up into blocks, maybe 50 images at a time. So you simply add a second outer loop over blocks.
The app is only a demo, and can be improved in many ways. One thing you would want to do if you ever released it, is ensure that it would process images even without xgrid running, locally. Another is that you would want to be able to locate controllers with rendezvous, query them to find out which has the most resources available, and submit to it.
Drew
-
Node List
2004-05-21 08:22:02 cyberassassin [Reply | View]
I hope this article continues, this is a very interesting example, and I can think of dozens of implementations....
One thing I would like to know how to do is the node drawer similiar to the drawer in Apple's XGrid BLAST app. The drawer tells you the number of available nodes (XGrid Agents), the number of working nodes, and the available aggregate CPU Power. This would make a great addition to the Photo Industry app. Maybe the next article.... Also interesting is the tachometer, or other graphic representation of cluster computing power....
Thanks for covering this. I find it some of the more fascinating stuff coming out of Apple, and hope to see your prediction of a XGrid API soon. And here's hoping for more articles relating to XGrid -
Node List
2004-05-25 11:32:55 drewmccormack [Reply | View]
You can get the information you need for the drawer using variousxgridoptions. Here is a bit of output to various commands I issued on my iBook:
drew ibook: xgrid -h localhost -node list
{
nodeIdentifierArray = ("Drew McCormack\U2019s Computer", DrewMcCormacksiBook);
nodeListIdentifier = Rendezvous;
}
drew ibook: xgrid -h localhost -node status -id DrewMcCormacksiBook
{
currentCPUPower = 0;
currentTaskCount = 0;
nodeName = DrewMcCormacksiBook;
nodeState = Available;
}
drew ibook: xgrid -h localhost -node info -id DrewMcCormacksiBook
{maximumCPUPower = 600; maximumTaskCount = 1; nodeName = DrewMcCormacksiBook; }
As you can see, you can list the nodes that a controller has access to. You can also get info about each node in the list, with-node info, and you can see whether a node is available with-node status.
You can use this info from Cocoa using NSTask. Note that output of these commands is in the form of property lists, so you can easily read the information into Cocoa using methods likedictionaryWithContentsOfFile:.
Hope this helps.
Drew






Also, if I had 10000 images, your app would create 10000 threads?