#duraspace IRC Log


IRC Log for 2012-03-01

Timestamps are in GMT/BST.

[0:09] * mdiggory (~mdiggory@ip72-199-216-7.sd.sd.cox.net) has joined #duraspace
[0:13] * helix84 (a@ has left #duraspace
[2:27] * hpottinger (~hpottinge@ has joined #duraspace
[2:57] * hpottinger (~hpottinge@ Quit (Quit: Later, taterz!)
[3:02] * mdiggory (~mdiggory@ip72-199-216-7.sd.sd.cox.net) Quit (Quit: mdiggory)
[6:56] -leguin.freenode.net- *** Looking up your hostname...
[6:56] -leguin.freenode.net- *** Checking Ident
[6:56] -leguin.freenode.net- *** Found your hostname
[6:57] -leguin.freenode.net- *** No Ident response
[6:57] * DuraLogBot (~PircBot@atlas.duraspace.org) has joined #duraspace
[6:57] * Topic is '[Welcome to DuraSpace - This channel is logged - http://irclogs.duraspace.org/]'
[6:57] * Set by cwilper!ad579d86@gateway/web/freenode/ip. on Fri Oct 22 01:19:41 UTC 2010
[13:02] * mhwood (mwood@mhw.ulib.iupui.edu) has joined #duraspace
[13:38] * elschlomo (~ruckus@HSI-KBW-046-005-164-187.hsi8.kabel-badenwuerttemberg.de) has joined #duraspace
[13:38] <elschlomo> hola
[13:52] * ajs6f (89369bb4@gateway/web/freenode/ip. has joined #duraspace
[13:56] * ajs6f (89369bb4@gateway/web/freenode/ip. Quit (Ping timeout: 245 seconds)
[13:57] * ajs6f (89360b5e@gateway/web/freenode/ip. has joined #duraspace
[14:00] * eddies (~eddies@cm126.sigma9.maxonline.com.sg) has joined #duraspace
[14:00] * eddies (~eddies@cm126.sigma9.maxonline.com.sg) Quit (Changing host)
[14:00] * eddies (~eddies@unaffiliated/eddies) has joined #duraspace
[14:04] <elschlomo> Chris is rolling out akubra 0.4 after over a year
[14:04] <elschlomo> probably this week
[14:05] <elschlomo> this involved updating dependenciy versions and adding some docs on github
[14:05] <elschlomo> *involves
[14:07] <elschlomo> the plan is changed to create a bugfix update 0.3.1 and do the rest for a nex release as suggested by eddie
[14:07] <eddies> https://jira.duraspace.org/browse/AKUBRA-7
[14:07] <kompewter> [ [#AKUBRA-7] FSBlobIdIterator escapes file paths - DuraSpace JIRA ] - https://jira.duraspace.org/browse/AKUBRA-7
[14:08] * EdAtTheAlliance (1809d1a2@gateway/web/freenode/ip. has joined #duraspace
[14:10] <elschlomo> Problem stems from reescaping already escaped characters on the way up from the akubra-fs layer
[14:10] <elschlomo> and the notion that handling *arbitrary* paths seems impractibal now
[14:14] <elschlomo> chris: akubra was intended for using file URIs only so the mapper from arbitrary URIs to files in the Blobstore is needed
[14:16] <elschlomo> there is now way to tell the differenc between the '+' symbol and a space on the way up from akubra, since they have both been encoded in the same character
[14:18] <elschlomo> frank was a bit slow and needed some explanation
[14:19] <elschlomo> the ways suggested are having people use only a subset of characters or some metadata about the encoding
[14:21] <elschlomo> eddie suggests having a way to tell akubra how paths were encoded and have akubra use those definitions for encoding/decoding to guarantuee symmtery
[14:24] <elschlomo> AKUBRA-7 will be coverted into a feature request, to see if interest from the community is there for arbitrary paths.
[14:25] <elschlomo> https://jira.duraspace.org/browse/FCREPO-1062
[14:25] <kompewter> [ [#FCREPO-1062] invalid XML datastream checksums after migration from inline (X) to managed (M) via migration script - DuraSpace JIRA ] - https://jira.duraspace.org/browse/FCREPO-1062
[14:28] <elschlomo> part one is the root of the problem: a bug which needs to be fixed by the committers, but how to handle the development of the tool to fix the problem for the short-term in running systems..
[14:31] * tdonohue (~tdonohue@c-67-177-108-221.hsd1.il.comcast.net) has joined #duraspace
[14:31] <elschlomo> there might be a general pattern with these migration related issues. see AKUBRA-7.
[14:31] * sbayliss (bcde58ad@gateway/web/freenode/ip. has joined #duraspace
[14:33] <elschlomo> the things eddie has done for AKUBRA-7 mitigation might be a template for a general solution of migration related issues
[14:34] <elschlomo> franks has dropped form the call it seems
[14:38] <elschlomo> there ougth to be a well defined canonililazion format which should be used for migrations
[14:39] <elschlomo> argg *canonicalization
[14:39] <ajs6f> Canonicalize that spelling! {grin}
[14:39] <elschlomo> sounds a bid drunk when iread it out loud ;)
[14:41] <elschlomo> bugs needs to be fixed, the migration utility should be changed and content holders will have to manipulate their collections
[14:43] <elschlomo> eddie will update the issue...
[14:44] <elschlomo> https://jira.duraspace.org/browse/FCREPO-748
[14:44] <kompewter> [ [#FCREPO-748] Ability to Authenticate to http-based storage when getting "E" datastreams - DuraSpace JIRA ] - https://jira.duraspace.org/browse/FCREPO-748
[14:45] <elschlomo> there is a PoC since some time but it's not yet merged into master
[14:45] <elschlomo> and there is need for a fix now in one of chris' projects
[14:47] <elschlomo> making fedora work with HTTP-BASIC Auth so it can itself authenticate to other services seems feasible
[14:47] <elschlomo> gor e.g. HTTP bases torages
[14:47] <elschlomo> *storages
[14:55] <elschlomo> for a first step it would be good to have fedora's http client use specific credentials, but in a secind step there should be some thought put into this issue and whether a protocol for such use-cases will have to defined.
[14:56] * ajs6f (89360b5e@gateway/web/freenode/ip. Quit (Ping timeout: 245 seconds)
[14:59] <elschlomo> there is a non testable but reproducable issue which seems occur with multiple threads (> 15)
[15:00] * ajs6f (89360b5e@gateway/web/freenode/ip. has joined #duraspace
[15:00] <elschlomo> that eddie has when ingesting data
[15:01] <elschlomo> there is threadweaver to have some control over concurrency issues in tests
[15:01] <elschlomo> it has been used in Akubra
[15:01] <sbayliss> Issues relating to DOManager#doCommit logging and clean-up:
[15:01] <sbayliss> https://jira.duraspace.org/browse/FCREPO-1029
[15:01] <kompewter> [ [#FCREPO-1029] Tidy up logging in DefaultDOManager.doCommit() - ensure errors are logged as such - DuraSpace JIRA ] - https://jira.duraspace.org/browse/FCREPO-1029
[15:01] <sbayliss> https://jira.duraspace.org/browse/FCREPO-1030
[15:01] <kompewter> [ [#FCREPO-1030] Better tidy-up code on ingest failure - DuraSpace JIRA ] - https://jira.duraspace.org/browse/FCREPO-1030
[15:02] <sbayliss> https://jira.duraspace.org/browse/FCREPO-1031
[15:02] <kompewter> [ [#FCREPO-1031] Introduce clean-up code in DefaultDOManager.doCommit() for operations other than ingest - DuraSpace JIRA ] - https://jira.duraspace.org/browse/FCREPO-1031
[15:04] <elschlomo> stephen suggests an extra bit of debug code that validates ingested objects after doCommit()
[15:04] <elschlomo> which might actually hide the bug ;=)
[15:04] <elschlomo> synchronization is hard they say...
[15:08] * sbayliss (bcde58ad@gateway/web/freenode/ip. Quit (Quit: Page closed)
[15:08] * ajs6f (89360b5e@gateway/web/freenode/ip. Quit (Quit: Page closed)
[15:08] <elschlomo> meeting has been wrapped up at 15:08 UTC
[15:15] * elschlomo (~ruckus@HSI-KBW-046-005-164-187.hsi8.kabel-badenwuerttemberg.de) Quit (Quit: BitchX: causing all sorts of havok!)
[16:28] * EdAtTheAlliance (1809d1a2@gateway/web/freenode/ip. Quit (Ping timeout: 245 seconds)
[16:47] * eddies (~eddies@unaffiliated/eddies) Quit (Quit: Leaving.)
[17:42] * bradmc (~bradmc@207-172-69-79.c3-0.smr-ubr3.sbo-smr.ma.static.cable.rcn.com) Quit (Quit: bradmc)
[17:43] * bradmc (~bradmc@207-172-69-79.c3-0.smr-ubr3.sbo-smr.ma.static.cable.rcn.com) has joined #duraspace
[18:54] * KevinVdV (~KevinVdV@d54C154B1.access.telenet.be) has joined #duraspace
[19:21] * hpottinger (~hpottinge@mu-162198.dhcp.missouri.edu) has joined #duraspace
[19:41] * hpottinger (~hpottinge@mu-162198.dhcp.missouri.edu) has left #duraspace
[19:50] <tdonohue> Hi all, reminder that Day #4 of our DSpace Developers Virtual Summit starts in ~10 minutes. Join us via Skype (and IRC backchannel): https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-02-27+-+Virtual+Summit
[19:50] <kompewter> [ DevMtg 2012-02-27 - Virtual Summit - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-02-27+-+Virtual+Summit
[19:56] * grahamtriggs (~Graham@cpc18-stev6-2-0-cust239.9-2.cable.virginmedia.com) has joined #duraspace
[19:59] * aschweer (~schweer@schweer.its.waikato.ac.nz) has joined #duraspace
[20:01] <tdonohue> Hi all, Virtual Summit will be getting started here shortly. Connection details up at: https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-02-27+-+Virtual+Summit#DevMtg2012-02-27-VirtualSummit-ConnectionInstructions
[20:01] <kompewter> [ DevMtg 2012-02-27 - Virtual Summit - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-02-27+-+Virtual+Summit#DevMtg2012-02-27-VirtualSummit-ConnectionInstructions
[20:03] <tdonohue> currently just a small group in the Virtual Summit call. Any others planning to join today: grahamtriggs? kshepherd? PeterDietz? scottatm?
[20:05] <mhwood> Someone is speaking but I can't hear much.
[20:05] <mhwood> Better!
[20:07] <tdonohue> Kevin talking about being interested in starting a prototype of Discovery with Elastic Search as a backend.
[20:09] <tdonohue> Andrea: May help to allay questions/concerns around Solr, if we have other options for running Discovery (i.e. Elastic Search). Tim also mentions it'd be a good 'proof-of-concept' that Discovery can run on something other than Solr.
[20:09] <tdonohue> Graham: there are some potential concerns about Solr performance (as index size grows). Could be interesting to try out other options like Elastic Search.
[20:12] <tdonohue> All in agreement that prototyping Discovery + Elastic Search. Kevin will keep us up-to-date. Will create a JIRA & GitHub to get started.
[20:13] * hpottinger (~hpottinge@ has joined #duraspace
[20:14] <tdonohue> https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-02-27+-+Virtual+Summit#DevMtg2012-02-27-VirtualSummit-PotentialDiscussionTopics
[20:14] <kompewter> [ DevMtg 2012-02-27 - Virtual Summit - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-02-27+-+Virtual+Summit#DevMtg2012-02-27-VirtualSummit-PotentialDiscussionTopics
[20:14] * mdiggory (~mdiggory@rrcs-74-87-47-114.west.biz.rr.com) has joined #duraspace
[20:15] <hpottinger> sorry, I'm late, Skype says the conference is offline?
[20:15] <tdonohue> no, we're on here. Try again?
[20:15] <mhwood> Not offline to me.
[20:16] <grahamtriggs> Tell Skype it's wrong
[20:16] <aschweer> try looking at the skype user profile for that contact, sometimes that makes it come back online
[20:16] <hpottinger> I'm in
[20:18] <tdonohue> http://help.github.com/import-from-subversion/
[20:18] <kompewter> [ Help.GitHub - Import from Subversion ] - http://help.github.com/import-from-subversion/
[20:22] <mdiggory> http://scm.dspace.org/svn/repo/dspace/trunk/
[20:22] <kompewter> [ repo - Revision 6957: /dspace/trunk ] - http://scm.dspace.org/svn/repo/dspace/trunk/
[20:23] <mdiggory> http://scm.dspace.org/svn/repo/dspace/trunk/dspace-swordv2/
[20:23] <kompewter> [ repo - Revision 6957: /dspace/trunk/dspace-swordv2 ] - http://scm.dspace.org/svn/repo/dspace/trunk/dspace-swordv2/
[20:23] <tdonohue> MarkD talking about Maven project organization in 'trunk' code
[20:24] <tdonohue> SWORD2 is organized slightly different -- doesn't have sub-projects for '-api' and '-webapp'. Rather, used the capabilities of Maven to build both JAR & WAR from one module
[20:24] <mdiggory> http://search.maven.org/#artifactdetails%7Corg.dspace%7Cdspace-swordv2%7C1.8.2%7Cwar
[20:24] <kompewter> [ The Central Repository Search Engine ] - http://search.maven.org/#artifactdetails%7Corg.dspace%7Cdspace-swordv2%7C1.8.2%7Cwar
[20:26] <tdonohue> Rather than having all these Maven subprojects for -api, -war, do we want to consolidate these into one project for each of these modules (e.g. XMLUI, JSPUI, etc.)
[20:26] <mdiggory> http://scm.dspace.org/svn/repo/dspace/trunk/dspace/modules/swordv2/pom.xml
[20:27] <mdiggory> <!-- DSpace Implementation of SWORDv2 Provider -->
[20:27] <mdiggory> <dependency>
[20:27] <mdiggory> <groupId>org.dspace</groupId>
[20:27] <mdiggory> <artifactId>dspace-swordv2</artifactId>
[20:27] <mdiggory> <type>jar</type>
[20:27] <mdiggory> <classifier>classes</classifier>
[20:27] <mdiggory> </dependency>
[20:27] <tdonohue> several agree that SWORD2 structure is a good "model" for future Maven projects. Discussing whether to reorganize existing projects -- would this affect user customizations?
[20:28] <tdonohue> Would this mess up existing "vendor drops"?
[20:29] <tdonohue> Can we document how this was changed, so that folks can expect this major reorganization in the next upgrade?
[20:30] <mhwood> Also document "this is how we organize the code now, look at e.g. swordv2 for examples."
[20:30] <tdonohue> good point, mhwood
[20:35] <tdonohue> https://wiki.duraspace.org/display/DSDOC18/Advanced+Customisation
[20:35] <kompewter> [ Advanced Customisation - DSpace 1.8 Documentation - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSDOC18/Advanced+Customisation
[20:37] <tdonohue> We need to document our now Maven project organization standards (likely under the "Advanced Customisation" section linked above).
[20:38] <tdonohue> s/now/new/
[20:38] <kompewter> tdonohue meant to say: We need to document our new Maven project organization standards (likely under the "Advanced Customisation" section linked above).
[20:44] * hpottinger wants to be sure that the oracle driver makes it over there somewhere... :-)
[20:45] <tdonohue> We should keep in mind our development environments (IDEs) and make sure this change doesn't affect it in any way.
[20:45] <tdonohue> beyond that, most seem in agreement of this simplification of the Maven modules (just needs docs and a bit of IDE testing)
[20:46] <KevinVdV> http://scm.dspace.org/svn/repo/dspace/trunk/dspace-discovery/
[20:46] <kompewter> [ repo - Revision 6957: /dspace/trunk/dspace-discovery ] - http://scm.dspace.org/svn/repo/dspace/trunk/dspace-discovery/
[20:51] <tdonohue> Talking about whether Discovery code needs to be reorged as well. MarkD says 'provider' part may be more appropriate alongside dspace-api eventually. And the 'xmlui' projects for Discovery obviously could be placed alongside XMLUI code itself
[20:51] <tdonohue> Graham: the question of reorganizing Discovery leads back to whether it becomes the "default" or the "only" option
[20:51] <PeterDietz> peter just jumped in
[20:51] <PeterDietz> (sorry for being away, staff meeting)
[20:57] <tdonohue> Graham points out that technically the Browse DB tables are "self-generating". As long as the Browse code still exists, then someone can choose to use the existing Browse DB tables (and system).
[20:59] <tdonohue> (A missed previous point -- reorganizing Discovery and splitting it up alongside API and XMLUI, could help to simplify the codebase in some ways -- less projects, maybe less confusion?)
[21:00] <tdonohue> Should Browse & Search even be part of DSpace-API? They are more UI related
[21:08] * hpottinger wishes for a white board...
[21:08] <tdonohue> The "pieces" that are DSpace: Content Model, Search/Browse index/tool, an Event Manager, a UI (or multiple)
[21:10] <PeterDietz> you could be (/dspace/bin/dspace import), and then OAI harvest.. to be a functional bare-bones
[21:10] <tdonohue> all are recalling Richard Rodger's talk at OR2008 (southhampton) about "DSpace 'bricks'"
[21:14] <grahamtriggs> http://dabbleboard.com/
[21:14] <kompewter> [ Dabbleboard - Online whiteboard for drawing & team collaboration - Interactive whiteboard software ] - http://dabbleboard.com/
[21:15] <tdonohue> MarkD reflects back on DSpace2 work that could be useful for some of these discussions
[21:17] <tdonohue> Tim wonders if it's time to rethink/reanalyze the 'pieces' that are DSpace (similar to the DSpace2 exercises)...obviously keeping in mind things that had been discussed/decided in the past
[21:17] <mdiggory> Heres a link to a proposal page for consolidating the Maven projects... still much to document here, its just a beginning https://wiki.duraspace.org/display/DSPACE/Maven+Project+Consolidation
[21:17] <kompewter> [ Maven Project Consolidation - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/Maven+Project+Consolidation
[21:20] * aschweer (~schweer@schweer.its.waikato.ac.nz) has left #duraspace
[21:20] * grahamtriggs (~Graham@cpc18-stev6-2-0-cust239.9-2.cable.virginmedia.com) Quit (Read error: Connection reset by peer)
[21:20] <hpottinger> anybody gets a dabbleboard they want me to look at, paste me a link :-)
[21:21] <tdonohue> General Summary: we're floating around a lot of similar ideas through many of these meetings. Much discussion revolves around reorg of DSpace (and perhaps even 'splitting' or 'combining' various 'pieces' as necessary).
[21:21] <tdonohue> Tim hopes we can start to even 'whiteboard' some of this up tomorrow (on Dabbleboard perhaps) and see where we can go with that...
[21:23] <mhwood> I think we've been doing something that comes quite naturally to people: preferring implementation details to stepping back and making sure we're all clear on the concepts to be embodied.
[21:23] <tdonohue> Also, looking for general feedback on this Virtual Summit -- would you want to do this again? (e.g. every 6 months? or on a timed basis) Has it been useful? Is a week too long? or too short? Any other general suggestions...
[21:23] <mhwood> I can do a week once or twice a year. More might be difficult.
[21:23] <tdonohue> mhwood -- good point. We do tend to go into implementation details (which is understandable, even I do that frequently)
[21:24] <hpottinger> frankly, I think it's been helpful, and personally, I've really appreciated the excuse to head home every afternoon ;-)
[21:25] <hpottinger> week or two a year would be doable for me
[21:28] <tdonohue> The other option admittedly is to shorten it to 2-3 days if we decided we wanted to have such discussions more frequently. It doesn't have to be a full week.
[21:28] * tdonohue realizes that tomorrow's meeting will be Friday night for some (in Europe) and Sat morning for some (NZ). That's the one hard thing about a full week
[21:29] <mhwood> Discussing stuff at this level, I think we need at least a quarter to work on the results of the discussion.
[21:29] <hpottinger> mdiggory, the writeup on the consolidation proposal is clear, thanks for posting it up, would a link to the Maven Skinny WARs page (http://maven.apache.org/plugins/maven-war-plugin/examples/skinny-wars.html) be helpful at all?
[21:30] <tdonohue> good point, mhwood. We don't want them to be too frequent, otherwise it will just be the same discussion over and over again. So, every 6 mos or 1 year might be more reasonable.
[21:30] <mdiggory> please add anything you think is relevant
[21:30] <mdiggory> that was for hpottinger
[21:31] * hpottinger hunts around for his red pen
[21:31] <mhwood> I do sometimes feel I'm playing the Ugly American, always having these meetings happen during the workday here while much of the world is on a much more inconvenient time.
[21:32] <mdiggory> Hey, I resent being called "Ugly"
[21:32] <hpottinger> perhaps rotate the timing, much as OR alternates between Europe and US
[21:32] <mdiggory> Even if I am, its impollite
[21:33] <mhwood> I thought I had written "I"....
[21:33] * hpottinger snickers, politely
[21:34] <mdiggory> Well, I wouldn't call you "Ugly", it'd be impolite too
[21:34] <mhwood> Anyway it's really late right now in Poland or India.
[21:34] <KevinVdV> 22:34 here in Belgium
[21:35] <mhwood> Yes, and that's not very far east at all.
[21:37] <mhwood> I see that Poland is also on MET. I don't have a good grasp of distances in Europe.
[21:39] <hpottinger> mdiggory: your wiki page has been "enriched"
[21:41] <KevinVdV> Needs to run I'll see you guys tomorrow
[21:41] * KevinVdV (~KevinVdV@d54C154B1.access.telenet.be) Quit (Quit: HydraIRC -> http://www.hydrairc.com <- Like it? Visit #hydrairc on EFNet)
[21:44] <mhwood> http://en.wikipedia.org/wiki/Ugly_American_%28epithet%29
[21:44] <kompewter> [ Ugly American (epithet) - Wikipedia, the free encyclopedia ] - http://en.wikipedia.org/wiki/Ugly_American_%28epithet%29
[21:49] <hpottinger> since we're talking about reorganizing maven stuff, I for one and really glad I decided to jump on the IntelliJ bandwagon. :-)
[21:50] <mhwood> I'm just happy that NetBeans doesn't seem to latch up for half an hour at a stretch like Eclipse used to. :-|
[22:03] * mhwood (mwood@mhw.ulib.iupui.edu) has left #duraspace
[22:20] * cbeer (cbeer@2600:3c03::f03c:91ff:fedf:a498) has joined #duraspace
[23:10] * tdonohue (~tdonohue@c-67-177-108-221.hsd1.il.comcast.net) Quit (Read error: Connection reset by peer)
[23:10] * hpottinger (~hpottinge@ Quit (Quit: Later, taterz!)
[23:13] * mdiggory (~mdiggory@rrcs-74-87-47-114.west.biz.rr.com) Quit (Quit: mdiggory)

These logs were automatically created by DuraLogBot on irc.freenode.net using the Java IRC LogBot.