Timestamps are in GMT/BST.
[0:45] * ymli (6fc0d7b8@gateway/web/freenode/ip.111.192.215.184) has joined #duraspace
[1:16] * ymli (6fc0d7b8@gateway/web/freenode/ip.111.192.215.184) has left #duraspace
[1:16] * ym111 (~ymli@111.192.215.184) has joined #duraspace
[4:08] * ym111 (~ymli@111.192.215.184) Quit ()
[6:28] -verne.freenode.net- *** Looking up your hostname...
[6:28] -verne.freenode.net- *** Checking Ident
[6:28] -verne.freenode.net- *** Found your hostname
[6:28] -verne.freenode.net- *** No Ident response
[6:28] * DuraLogBot (~PircBot@atlas.duraspace.org) has joined #duraspace
[6:28] * Topic is '[Welcome to DuraSpace - This channel is logged - http://irclogs.duraspace.org/]'
[6:28] * Set by cwilper!ad579d86@gateway/web/freenode/ip.173.87.157.134 on Fri Oct 22 01:19:41 UTC 2010
[12:13] * mhwood (mwood@mhw.ulib.iupui.edu) has joined #duraspace
[13:33] * tdonohue (~tdonohue@c-67-177-108-221.hsd1.il.comcast.net) has joined #duraspace
[17:01] <tdonohue> Hi all, if anyone has DSpace topics to discuss, my office hours are for the next 3 hours : https://wiki.duraspace.org/display/~tdonohue/DSpace+Office+Hours
[19:49] * hpottinger (~hpottinge@mu-162198.dhcp.missouri.edu) has joined #duraspace
[19:52] <tdonohue> Hi all, we resume our weekly DSpace Devel Mtgs at the top of the hour (~10mins) : https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-07-18
[19:52] * helix84 (a@195.113.97.174) has joined #duraspace
[19:53] * kompewter (~kompewter@ec2-50-17-201-82.compute-1.amazonaws.com) has joined #duraspace
[19:53] * mdiggory (~mdiggory@rrcs-74-87-47-114.west.biz.rr.com) has joined #duraspace
[20:00] * richardrodgers (~richardro@18.189.24.43) has joined #duraspace
[20:00] <tdonohue> Hi all, it's now time for our weekly DSpace Devel Mtg: https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-07-18
[20:00] <kompewter> [ DevMtg 2012-07-18 - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-07-18
[20:01] * robint (52292725@gateway/web/freenode/ip.82.41.39.37) has joined #duraspace
[20:01] <tdonohue> we'll kick things off with some JIRA catchup (for about 15 mins): https://jira.duraspace.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+DS+AND+resolution+%3D+Unresolved+AND+Key%3E%3DDS-1073+ORDER+BY+key+ASC
[20:01] <kompewter> [ Issue Navigator - DuraSpace JIRA ] - https://jira.duraspace.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+DS+AND+resolution+%3D+Unresolved+AND+Key%3E%3DDS-1073+ORDER+BY+key+ASC
[20:01] <kompewter> [ https://jira.duraspace.org/browse/DS-1073 ] - [#DS-1073] The maximum flag on filter-media is useless if results are returned in the same order every time - DuraSpace JIRA
[20:01] <tdonohue> starting with DS-1073
[20:01] <kompewter> [ https://jira.duraspace.org/browse/DS-1073 ] - [#DS-1073] The maximum flag on filter-media is useless if results are returned in the same order every time - DuraSpace JIRA
[20:04] <mhwood> Looks like richardrodgers had something brewing that addresses this?
[20:04] * lyncode (~DSpace@bl23-51-244.dsl.telepac.pt) has joined #duraspace
[20:06] * sands (~sandsfish@18.189.31.239) has joined #duraspace
[20:06] <richardrodgers> mhwood: yes, in a sense. There is a set of curation tasks for 1.81 that can replace media filter
[20:06] <richardrodgers> sorry 1.8+
[20:06] <tdonohue> my mind is hazy here -- did we release those in 1.8? Or is this an "add-on" of curation tasks?
[20:07] <richardrodgers> tdonohue: the media filter tasks were not released with 1.8, they are an add-on (that need some testing & hardening BTW)
[20:08] <tdonohue> ok. I see. Are their plans to release then? Just trying to determine if we should link this Ds-1073 off to another ticket around those media filter tasks?
[20:09] <richardrodgers> sure - it might make sense to link to a JIRA for those...
[20:09] <mdiggory> Seems like the order by is a much simpler change
[20:10] <mhwood> [PHB] Let's do both!
[20:10] <tdonohue> volunteers? Feel free to grab the ticket! :)
[20:11] <richardrodgers> sure, but I think (as mhwood points out - there may be performance issues)
[20:11] <tdonohue> at the very least, I like the idea of adding a JIRA ticket around the Curation Task MediaFilters & linking it as related to Ds-1073
[20:12] <sands> sorry, jumped in late. is that testing something we should make sure is on the 3.0 release test-a-thon list?
[20:12] <sands> (performance testing, et al)
[20:14] * sands throws a wrench in the gears
[20:14] <mdiggory> I don't see much difference between sorting the results and processing only the new items vs creating a queue to list new items, the later actually sounds more complex.
[20:14] <hpottinger> I think the curation task media filters is a tidy solution to the problem, providing the tooling for processing on ingest, as well as updating the media filter process in general. I don't really have concerns about performance issues for a nightly cron job, as long as it finishes at some point, and is reasonably efficient at what it's doing.
[20:15] <mhwood> Scalability. If you add ten new items a day to a collection of a trillion, you only process ten things vs. one trillion every night.
[20:15] <mdiggory> "lastmodified" > yesterday
[20:15] <richardrodgers> mdiggory: there is a huge performance difference, if thumbnails (e.g.) are wired into a workflow step, only *one* item is run at a time, only once
[20:18] <tdonohue> Is there a resolution here then? Sounds like more favor linking this off to curation task media filters (can someone create that new ticket?)
[20:18] <PeterDietz> So.. for non-workflow submissions, where do they get queued? (batch import / sword)
[20:19] <PeterDietz> I do like curation post-submission, and I like media-filter where last-modified > yesterday
[20:19] <mhwood> That's why I keep saying that there should be no non-workflow submissions; only workflows with no interactive steps.
[20:19] <mdiggory> Sounds like an Action or Automated Step in the Configurable Workflow to me, or a Consumer processing on InstallItem, I don't specifically see a benefit of using a Curation task for this.
[20:20] <tdonohue> "media-filter where last-modified > yesterday" is making an assumption that everyone is running this cronjob nightly. So, I think it'd need to be "media-filter where last-modified > [date]"
[20:20] <richardrodgers> mdiggory: the advantage is that one can use the same code in the admin UI, or in batch etc, On demand, cron-scheduled, etc
[20:20] <mdiggory> But the point was just to make MediaFilter operate more effectively on its list of items.
[20:20] * tdonohue notes we're digging really deep here ... not sure this is the best usage of our time?
[20:20] <richardrodgers> and it does, the list is reduced to 1
[20:21] <tdonohue> perhaps we need someone(s) to investigate & come back with solution/proposed solution?
[20:21] <mhwood> The point was that this list of items is "all items in the repository". Bad for scaling.
[20:22] <mhwood> tdonohue: yes, there's not enough agreement to deal with 1073 now.
[20:23] <mdiggory> I'm not challenging the use of the curation system specifically for curation tasks in workflow, I'm just not a big fan of using as a hammer everywhere in dspace
[20:24] <tdonohue> anyone interested in this area/ticket? Or do we just have a lot of opinions :)
[20:24] <hpottinger> I think the proposed link to the Jira ticket for the media filter curation tasks is a good first step, and may actually provide an answer for 1073 at some point
[20:25] <tdonohue> ok. moving on. We're done with JIRA for today. Ds-1073 needs a volunteer at some point. We also need a ticket for "media filter curation tasks".
[20:25] <richardrodgers> I'll open a ticket
[20:26] <mdiggory> Anyhow, the problem with FilterMedia specifically is not solved by using the Curation system. Filtermedia sorting and selecting the items that need to be processed by a MediaFilter implementation would still benefit by some minor changes to optimize selecting items. Even if it was a curation task that was executing whatever action was to be completed.
[20:26] <mhwood> I can look into improving the current touch-every-item scheme.
[20:26] <tdonohue> Ok. next topic: Followup from OR12 Developer Meeting. I just wanted to leave some time here for any necessary followups we need to do. Since I wasn't there, this means I'm looking to all you who were to prompt us in what things need to happen (future topics to discuss, mtgs to schedule, etc.)
[20:27] <tdonohue> was there anything specific that we want to bring up out of the OR12 Mtg? Major topics/discussions? Future Special Topics mtgs we should have?
[20:27] <hpottinger> still digging through notes, am planning to post more to the page
[20:28] <tdonohue> Some folks have started some OR12 Mtg notes here: https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-07-09+-+OR12+Meeting#DevMtg2012-07-09-OR12Meeting-MeetingNotes (Thanks robint & hpottinger!)
[20:28] <kompewter> [ DevMtg 2012-07-09 - OR12 Meeting - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2012-07-09+-+OR12+Meeting#DevMtg2012-07-09-OR12Meeting-MeetingNotes
[20:29] <hpottinger> though, two highlights (which I've added already) are mdiggory's link to the ThinkupApp's Git workflow, and the proposed ongoing joint DCAT and committers meeting
[20:29] <robint> I think we need to discuss and clarify what a distribution of DSpace entails
[20:30] <robint> I'm thinking about the rest-api and XOAI (hello lyncode !)
[20:30] <tdonohue> robint -- that sounds like something that would be highly important to figure out soon. Special Topic meeting perhaps in the next week or two?
[20:31] <hpottinger> and the notion of the need for some kind of current development project portal/clearinghouse, perhaps automated with RSS feeds from various other organizational wikis and/or issue trackers...
[20:31] <tdonohue> hpottinger -- was there any planning around the "proposed ongoing joint DCAT & Committers mtg", e.g. where/when? Or is that something I should just chat with Val about (she's on vacation/holiday till end of the month, but I can followup with her afterwards)
[20:31] <helix84> robint: you mean something like fedora spins? (not fedora commons, fedora the cousin of Red Hat)
[20:32] <robint> helix84: We just need to figure out a good way to make modules available without packaging everything
[20:33] <mdiggory> robint: release 3.0 first, then release versions of the modules
[20:33] <helix84> robint: i had some thoughts in this area, too. was there any discussion about this in the OR12 meeting?
[20:33] <hpottinger> tdonohue: no planning, other than a concern raised that "it can't just be IRC" DCAT members feel disoriented by the lack of our shared "exformation"
[20:33] <mdiggory> provide instructions on how to add the modules to dspace build
[20:33] <robint> mdiggory: what is in 3.0 ?
[20:33] <tdonohue> hpottinger -- ok, I'll followup with Val when she gets back.
[20:34] <mdiggory> I thought you were asking how to release addons separate from 3.0, sorry
[20:34] <robint> mdiggory: yes you are right
[20:34] <robint> Just not sure what is an addon and what is 'core'
[20:35] <mhwood> "Add modules to DSpace build" is the problem. People shouldn't be building DSpace to add modules; they should be unpacking built modules on top of an installed DSpace. But there's no infrastructure yet for doing that.
[20:35] <mdiggory> dspace-rest could be released properly in parallel just after dspace 3.0, comments could be added to the poms with instructions on how to enable it.
[20:35] <mdiggory> mhwood: basically we are saying the same thing...
[20:36] <helix84> mdiggory: how do i know i should look in poms for instructions? how is that better from a wiki page (current approach)?
[20:36] <tdonohue> Right, so the big question is are we still releasing *everything* in one big download/package. Where everything = JSPUI, XMLUI, OAI, SWORD(1 & 2), SOLR, LNI, etc.. Or, do we start to narrow our distribution & move more modules into "addons"
[20:36] <richardrodgers> +1 mhwood that is the model we should be working towards (mds takes a crack at this)
[20:36] <tdonohue> but, as mhwood pointed out, we don't have a way (infrastructure) to do that yet
[20:37] <mdiggory> tdonohue: thats a rabbit hole we just crawled out of last year, are you sure we want to fall back into it again?
[20:37] <tdonohue> mdiggory -- that's the question at hand right now. That's what is meant by a "distribution". robint brought it up, not I :)
[20:37] <hpottinger> perhaps a distraction will work? :-) back on to OR12 meeting, my notes get very scattered during the "hot topics" section, lots of lines and arrows :-) but I can say there was a good deal of discussion of the details of what "metadata for all" means, and various ideas on implementing it
[20:38] <lyncode> tdonohue: most DSpace distributions do not use all dspace modules, so i would agree with the 2nd choice
[20:38] <mdiggory> what is meant by distribution is what we did for 1.8 because async release is not feasible unless everyone embraces individual releases of individual modules (api, jspui, xmlui, etc)
[20:39] <lyncode> most DSpace use cases*
[20:40] <mdiggory> and TBH, I don't blame us for not wanting to go down that route either, there is an increase in complexity that I don't think the developer community can handle
[20:40] <helix84> how about distributing everything but forcing users to choose what they want? like <pseudocode>mvn package -P xmlui,oai,swordv2,rest</pseudocode>
[20:41] <mdiggory> I would say that the maven project consolidation will reduce the number of modules significantly (eliminating dspace-stats and a number of the dspace-discovery modules by absorbing them into dspace-api and dspace-xmlui)
[20:41] <mhwood> People shouldn't be running mvn or even necessarily have it unless they are modifying code. But again, we're not there yet.
[20:41] <sands> helix, that does bar us from what mdiggory suggested, which is the ability to add modules without making them part of the build.
[20:41] <sands> mhwood: agreed.
[20:42] <sands> i don't think documentation needs to be in poms.
[20:42] <sands> i think there just needs to be documentation about which pom section to uncomment, (or add to)
[20:42] <sands> in the absence of infrastructure
[20:42] <mdiggory> modules need to be part of the maven build not specifically to compile them, but to bring them into the wars as dependencies and/or overlays
[20:42] <sands> mdiggory: that's one way to envision modules
[20:43] <sands> or maybe i'm talking more about "add-ons". there's a fine distinction there i think.
[20:43] <helix84> i think lyncode solved this nicely with his XOAI module (next agenda point) - he provides it with a nice and easy install script, you can look at it here: http://www.lyncode.com/dspace/addons/xoai/
[20:43] <kompewter> [ Lyncode ] - http://www.lyncode.com/dspace/addons/xoai/
[20:43] <tdonohue> So, it sounds like we are getting around to the idea that, unfortunately, we don't have infrastructure to support anything "fancy". So, this would imply that 3.0 distribution will look much like 1.8 (with everything together), unless someone comes up with a bright idea before "feature freeze"
[20:44] <mdiggory> sands: unless you want to engineer the complexity of an "addon mechanism" for dspace that operates like some OSGI container, the maven build process is the only mechanism we currently have to include an addon. I'm not being hypothetical
[20:44] <helix84> maybe all that's needed is to create a simple packager which will produce such install scripts (it's based on diff)
[20:44] <mhwood> The addon inclusion tool should be something like 'tar' or 'unzip'.
[20:44] <sands> tdonohue: yes, we'd need the bright idea, but also modules would have to implement said bright idea. not sure if there's time for that. though if we're talking async releases of "add-ons", maybe not. (imagineering)
[20:45] <helix84> if you're an addon author, you'll create a package for the last stable dspace version
[20:45] <sands> mdiggory: i'm not clear on the requirement for overlays/war includes.
[20:45] <sands> helix84: yup
[20:46] <sands> i'm sure there's more complexity for certain use cases, but if we're talking quick and dirty, we probably won't be able to cover every use case in the short term.
[20:46] <tdonohue> lyncode's little install script for XOAI does look interesting, helix84. The only small problem I do see is that it's Linux/Unix specific. But, I'm sure we could build something similar for Windows folks.
[20:46] <mdiggory> sands: dspace/modules/xmlui is an overlay of dspace-xmlui-webapp, the requirement is intrinsic
[20:46] <mdiggory> Now for the hypothetical.... The Maven "Assembly" process is currently a fairly typical strategy taken from the maven community...
[20:47] <mdiggory> the Assembly Process doesn't neccessarily mean "compile"
[20:47] <mdiggory> and/or "package"
[20:48] <mdiggory> It may be possible that we can extend the assembly process to not use overlays in dspace/modules
[20:49] <mdiggory> but instead pull the webapp that placed into target/dspace-build..../webapps from maven central
[20:49] <helix84> what about the way translations are pulled in during the maven step? could this work for whole modules?
[20:49] <richardrodgers> what mds does is use maven assembly to create installable zip files of add-ons…separating authoring time from install time
[20:49] <sands> tdonohue, i'm curious about XOAI's approach to install. where do i look to see that script?
[20:49] <mdiggory> helix84: thats part of the overlay mechanism...
[20:50] <mdiggory> what does an installable zip provide?
[20:50] <helix84> sands: i linked it above. download the file.
[20:50] <tdonohue> sands -- helix84 shared the link to the XOAI's install script above -- http://www.lyncode.com/dspace/addons/xoai/ Just download the Zip file & unzip it & you'll see what it's doing
[20:50] <kompewter> [ Lyncode ] - http://www.lyncode.com/dspace/addons/xoai/
[20:50] <richardrodgers> jars, config files, war files, etc
[20:50] <mdiggory> richardrodgers, will it resolve dependency conflicts?
[20:51] <richardrodgers> mdiggory: I think so (I capture maven dependency info from build time at keep it in the zip, so it's available at install)
[20:52] <richardrodgers> at ->and
[20:52] <mdiggory> is the install actually running maven then?
[20:52] <richardrodgers> no, nor ant
[20:52] <mhwood> If we're writing robust code, dependency conflicts should only occur between e.g. 1.x and 2.x of a dependency.
[20:53] <tdonohue> so, do we want to continue this thread, or stop & discuss XOAI a bit?
[20:53] <tdonohue> (notes we are running short on time)
[20:54] <mdiggory> richardrodgers: thats my biggest concern, if my addon requires a newer version of dependency.jar, maven makes sure thats the one in the lib instead of the original designated by dspace, its the greatest benefit we are getting by the maven approach.
[20:55] <mhwood> If your addon requires a newer version of something that comes in DSpace x.y.z then it isn't compatible with x.y.z.
[20:56] <lyncode> we (lyncode) are studying various ways of making dspace "addon aware", soon we will share that with the community
[20:56] <mhwood> ...and people will have to wait for the release of DSpace that you're developing against.
[20:57] <richardrodgers> yes, mhwood summarizes the approach, is add-on compatible with current install vs 'make me a new system'
[20:57] <mdiggory> mhwood: richardrodgers thats not the practice that currently in place in the community
[20:57] <mhwood> That's a long-standing quarrel I have with the Java community in general. :-/
[20:58] <mdiggory> and IMO, its incredibly restrictive
[20:58] <tdonohue> I just want to state publicly that I for one *love* Maven's dependency management, but firmly *hate* the fact that it's required to install DSpace. No other major software (that I'm aware of) uses Maven as an installer. We really shouldn't be either (though I also firmly admit this is not something we can fix before 3.0)
[20:59] <mdiggory> we already have to release our addons for almost every minor release of dspace, if we have to wait for a release of dspace with the same dpendencies as our addon, that would have zero tractability
[20:59] <richardrodgers> mdiggory: agreed, but that it the only way to get real modularity. If I have to rebuild all of Eclipse (e.g.) and all its dependencies to add a plug-in, it wouldn't work too well
[20:59] * tdonohue says ditto for Ant
[21:00] <mhwood> No, you should only have to wait for a release of DSpace that uses the needed MAJOR VERSION (1.x vs. 2.x) of a dependency.
[21:00] <sands> tdonohue: agreed on using build tools for installation.
[21:00] <helix84> mdiggory: the other onption is to make a distribution of your addon with latest unreleased dspace. but that has its own disadvantages like untested code
[21:01] <hpottinger> tdonohue: I think of DSpace's reliance on Maven as an installer-builder as the "siren call" for developers. :-)
[21:03] <tdonohue> hpottinger: to be clear, I'm not against using Maven. I do like Maven to build DSpace. I just don't like it for *installation*. Normal users should never have to compile our code to install it. They should just run an install script or load a webapp in order to get DSpace setup. Power Users/Developers can still build from source if they really wanted.
[21:04] <helix84> mdiggory: the problem is the same as with linux distributions - there are two types - ones issuing regular stable releases vs. ones doing rolling releases
[21:04] <mdiggory> I'm sorry if I sound cynical... Its just hat we had this dialog already and its lasted almost half a decade now... theres what works now and hypothesis about what could be done thats better... proof is necessary to consider replacement, especially if it is going to have a large imact on existing users
[21:05] <mhwood> Argh, I've got to go. 'bye....
[21:05] <mdiggory> At OR12 Chris Wilper chated with many of us about wanting to revision Fedora...
[21:05] * mhwood (mwood@mhw.ulib.iupui.edu) has left #duraspace
[21:05] * tdonohue notes we're into "overtime". I'll stick around here though for a while. If you have to head out feel free
[21:05] <mdiggory> and based on all their experience with OSGI and DuraCloud.... they were leaning towards that as a solution...
[21:05] <helix84> mdiggory: what do you mean by "revision"?
[21:06] <mdiggory> "redo" Fedors
[21:06] <mdiggory> "redo" Fedora
[21:06] <tdonohue> mdiggory -- yea, Fedora folks are looking to "re-envision" Fedora. There are several areas of the software they've run into issues, and are looking towards "what is next"
[21:06] <mdiggory> I'm just trying to make a point....
[21:07] <tdonohue> mdiggory what is "that as a solution"? What are you referring to?
[21:07] <helix84> revision in what way? (there were several points made and i'm not sure which one you're referring to)
[21:07] <mdiggory> Unless there is someone in the community who is going to do the legwork to design, prototype and present a solution to the developer group for review, not much is going to happen here.
[21:08] <mdiggory> and more importantly
[21:08] <richardrodgers> mdiggory: check out mds - I think it does all this now
[21:08] <mdiggory> unless the developer group IS CONSULTED on the design and their criticisms taken into consideration during that design process.
[21:08] <mdiggory> the risk is that there will be opposition to the final product...
[21:09] <mdiggory> and I don't think we as a group want to see people wasting efffort in that manner
[21:09] <mdiggory> Iespecially when community resources are so limited
[21:10] <mdiggory> richardrodgers: it needs presenting...
[21:10] <mdiggory> there is a lot going on in mds.
[21:11] <mdiggory> I'm not trying to be a hard a$$ here... I'm trying create awareness
[21:11] <richardrodgers> https://github.com/richardrodgers/mds/wiki/Modules
[21:11] <kompewter> [ Modules · richardrodgers/mds Wiki · GitHub ] - https://github.com/richardrodgers/mds/wiki/Modules
[21:11] <helix84> yea, my first thought about mds was why is it all in one heap? it would be better to use branches to develop the individual improvements
[21:11] <mdiggory> we got burned this way with the 2.0 work...
[21:12] <tdonohue> mdiggory -- DSpace could also start to investigate the idea of forming "managed projects". That's what the Fedora Commons folks are looking at doing -- putting calls out for volunteers & funding geared specifically for a particular larger project (which may not be as easily implemented without such direct funding/support, cause of the lack of time that individual committers have)
[21:13] <helix84> it's just easier to review and agree/disagree on smaller improvements than to look at a heavily modified whole
[21:14] <helix84> richardrodgers?
[21:14] <sands> (he's on the phone for a second.)
[21:14] <richardrodgers> helix84: for some parts, yes - build mechanics are independent of other things
[21:15] <mdiggory> richardrodgers: Why not present it in the development proposals section of the dspace wiki, the way we've been trying to get folks to do so we know whats out there to discuss? I think there would have been earlier knowledge of this if it was under the DSpace umbrella
[21:16] <mdiggory> some of its ok... some of it, I'm cautious about...
[21:16] <richardrodgers> mdiggory: PeterDietz and others thought that proximate documentation in Github was prefereble
[21:16] <richardrodgers> preferable
[21:16] <mdiggory> but whose watching all these local forks out there????
[21:16] <PeterDietz> I do like having a certain amount of documentation right there in the github repo
[21:17] <PeterDietz> certainly having the installation / getting started.
[21:17] <helix84> this boils down to development model, e.g. as mark mentioned in http://thinkupapp.com/docs/contribute/developers/devfromsource.html
[21:17] <kompewter> [ Develop from Source — ThinkUp 1.0.8.1 documentation ] - http://thinkupapp.com/docs/contribute/developers/devfromsource.html
[21:17] * Hugo (~Hugo@a89-152-96-76.cpe.netcabo.pt) has joined #duraspace
[21:17] <PeterDietz> Then, I'm not sure where to weigh in on the break-down of having the full functionality documentation in detail to also live in github, or to push that over to confluence
[21:18] <helix84> basically do one thing in one branch and when ready, ask for review (e.g. on mailing lists)
[21:18] * Hugo is now known as Guest55730
[21:18] <helix84> i don't think _where_ the docs are is important as long as you notify us there's work you want us to review
[21:19] <richardrodgers> helix84: I have invited review from the start - and Github has notification tools galore - what would you suggest?
[21:19] <mdiggory> I think we need to make it clear that the onus is in the contributor to present the contribution to the community, not for the community to have to go visiting the contributor... its not that the document is in github or the wiki, but that its presented in a manner that everyone can have an opportunity to have a say
[21:20] <helix84> a) message on mailing list - please review this feature in this branch, docs are here
[21:20] <mdiggory> the wiki for documentation has the added benefit that your already in the same format as the final documentation
[21:20] <helix84> b) pull request
[21:20] * robint (52292725@gateway/web/freenode/ip.82.41.39.37) Quit (Ping timeout: 245 seconds)
[21:21] <mdiggory> helix84: arguing back the other way... I think Richards work predates the final DSpace github setup
[21:22] <helix84> i'm not saying it's easy, i'm trying to get more eyes to look at XOAI the way i described and noone looked so far
[21:22] <mdiggory> and in some ways isn't exactly viable for a "Pull request"
[21:22] <helix84> noted
[21:22] <richardrodgers> helix84: the mds code has altered space content api in several ways - pull requests won't function as I think you intend
[21:22] <tdonohue> I think one of the issues here is that GitHub doesn't notify us when you update something in mds, richardrodgers (Or at least not via email). So, even though I do "watch" MDS, I actually forget about it at times as I don't get update emails, etc. So, when I finally remember I have to dig through code/commits to try and figure out what changed recently.
[21:23] <mdiggory> richardrodgers: in terms of the alterations, I feel like there needs to be something in the middle that is not quite as controversial a restructure
[21:23] <helix84> let's approach it from a different angle: what do you think needs to be done to get your work into master?
[21:23] <mdiggory> if I understand "kernel" = "dspace-api"
[21:24] <mdiggory> then theres no need for that dramatic change.
[21:24] <helix84> tdonohue: yes, i have the same problem, too. i'll file an issue with github's software.
[21:24] <mdiggory> your just adding some capabilities to dspace-api
[21:24] <mdiggory> helix84: yea
[21:25] <richardrodgers> helix84: mds contains a number of experimental extensions - many could be 'adapted' to similar function in master
[21:26] <helix84> so you're not sure yourself if it's ready?
[21:26] <helix84> i don't mean for lack if testing
[21:26] <helix84> s/if/of/
[21:26] <kompewter> helix84 meant to say: i don't mean for lack of testing
[21:26] <richardrodgers> helix84: no, almost none of it is production-tested, it's a laboratory setting
[21:27] <sands> perhaps interested parties could (virtually, google+ hangout, or something) sit down with richardrodgers and walk through the changes. some way to get a better understanding of the approach.
[21:27] <mdiggory> richardrodgers: theres so much in here... I agree that to be able to get parts into master... work needs to happen to make smaller units of change.
[21:27] <helix84> but at some stage you have to decide - either it's the right way or the wrong way
[21:27] <mdiggory> and to recognize that theres overlap with some of it and other community member contributions
[21:27] <helix84> for each individual improvement
[21:27] <mdiggory> Pluggable Search... Discovery with pluggable engines.
[21:28] <helix84> i mean either you want your improvement to go to master or it's a dead-end
[21:28] <richardrodgers> yes, helix84 that's why one can download, install and play with it oneself
[21:28] <mdiggory> PluginManager Unplugged == greater adoption of Spring in a more conventional sense
[21:28] <richardrodgers> yes mdiggory all of those things
[21:29] <hpottinger> I think MDS is not quite a design proposal, as much as it is a scratch pad for potential design proposals, is my understanding correct, richardrodgers?
[21:30] <richardrodgers> hpottinger: it is a collection of design proposals that hang together in a DSpace frame.
[21:30] <helix84> i know i can do that. what i'm getting at is that your model of a single heap makes it easy to do the experiments. on the other hand, it makes it difficult to extract specific successful work from the rest.
[21:31] <hpottinger> it looks like fun, and it's jumped to the top of my pile of "things to play with"
[21:31] * Guest55730 (~Hugo@a89-152-96-76.cpe.netcabo.pt) has left #duraspace
[21:32] <tdonohue> I'll admit, helix84 is asking a lot of the same stuff I've asked richardrodgers myself recently about MDS. I also do wonder when it (a) starts to come back into DSpace (as a whole or in smaller pieces) or (b) starts to become something totally "un-DSpace" or something else entirely. Don't take it the wrong way, richardrodgers -- it's a very interesting experiment. But, I do think helix84 is hitting on some good points here..
[21:32] * HugoRibeiro (~Hugo@a89-152-96-76.cpe.netcabo.pt) has joined #duraspace
[21:32] <tdonohue> +1 helix84 (on the "single heap" point)
[21:33] <richardrodgers> helix84: yes, that is a complicated trade-off. I was trying to learn lessons from the DSpace 2.0 work, and keep it close to a deployable (minimal) whole system, rather than a set of design 'sketches'
[21:33] <mdiggory> just to caution, if the distance (in effort and resources) between the dspace master and mds proposal is too great to be bridged with our current resources... the exercise is not going to be very beneficial to the community, not any more than a GSoC project currently may be
[21:33] <tdonohue> (I'm not trying to make richardrodgers do even more work here though -- it's just a matter of trying to determine if there's ways we can even *help* to separate pieces out that can be brought back into DSpace as Pull Requests)
[21:34] <mdiggory> 2.0 was deployable and there was a demo system released
[21:34] <mdiggory> it just did not have production grade capabilities at the time funding ended
[21:35] <richardrodgers> mdiggory: it varies with parts of the system. For example, the build/module system is entirely at the pom/maven layer (plus one DSpace installer class),
[21:35] <richardrodgers> so the 'distance' is not great at all
[21:36] <richardrodgers> In other cases (e.g. data model refactor), the distance is larger
[21:37] <mdiggory> tdonohue: IMO, if a contribution is presented as a solution to problems being discussed in the community, what we've learned so far is that we need the contributor to present the work in a manner that can be easily applied to the current codebase.
[21:37] <mdiggory> we ask this of our contributors, why not of ourselves?
[21:37] <tdonohue> richardrodgers: is there any way to "document" or make it more evident which pieces are "usable" or nearly usable (with some minor tweaks) in DSpace? Just curious if we could do a more concentrated review on some pieces (if there's anything "ready" for an early review at this point in time)
[21:37] <helix84> i was not watching development at the time dspace 2.0 was brewing, so take my words with a grain of salt, but i think dspace 2.0 as a whole failed for the same reason. it was a "single heap". there comes time of review - what you can do is accept or reject. it's more likely there's parts you don't like in a large heap, so you're more likely to reject it to the point it never gets done. ergo smaller pieces of work are more likely to get accepted.
[21:39] <tdonohue> DSpace 2.0 was essentially a "single heap", that is true. It also was mostly implemented in a fashion where the majority of the committers unfortunately "fell out of the loop" and had major catchup to do at the end / had to dig through that "single heap" themselves with little formal documentation.
[21:40] <richardrodgers> tdonohue: I'll try to sketch out some 'low-hanging fruit' in mds that might help. (GItHub wiki page?)
[21:40] <tdonohue> (I don't mean anyone to take offense to that previous statement about 2.0 -- that's strictly my own opinion. )
[21:40] <sands> sorry everyone, gotta run. there is clearly a lot to talk about surrounding this topic. i hope we can have a follow-up, in this venue, or another.
[21:40] <sands> cheers
[21:40] * sands (~sandsfish@18.189.31.239) Quit (Quit: sands)
[21:40] <mdiggory> richardrodgers: can you not produce a clean fork of dspace in your user account and apply some of these changes so that they can be applied back as Pull requests and tested indivdually?
[21:41] <helix84> that would have the additional benefit of rebasing it on top of current HEAD
[21:41] <tdonohue> richardrodgers -- whereever you sketch that up is really your decision. Just let us know where it's at (dspace-devel or dspace-commit). You could also create a clean fork (separate project) like mdiggory suggests if you had the inclination, but that's really up to you.
[21:42] <mdiggory> of the github contribution strategies that we've been cooking up
[21:42] <mdiggory> prefix that with "it would also be a good test"
[21:42] <tdonohue> Where are these "github contribution strategies"? :) How can I "catchup" with what has been discussed?
[21:43] <mdiggory> tdonohue: you wrote them?!
[21:43] <richardrodgers> mdiggory: which changes? If, for example the changes in the content API, it would break all the business logic and UI code
[21:43] <mdiggory> https://wiki.duraspace.org/display/DSPACE/Development+with+Git
[21:43] <kompewter> [ Development with Git - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/Development+with+Git
[21:43] <helix84> right, tdonohue linked to some and mark linked to another
[21:44] <mdiggory> see Contributing Changes/Patches to DSpace via GitHub
[21:44] <tdonohue> mdiggory: Oh..you decided on what I wrote up about Git. Ok, cool. :) I don't know what you discussed at OR12, sorry -- it's not very clear in the OR12 Mtg notes.
[21:44] <mdiggory> Yea.... I thought someone else was taking notes ;-)
[21:45] <mdiggory> I talked about forking repos and using pull requests...
[21:45] <mdiggory> and I talked about the JIRA workflow....
[21:45] <hpottinger> I think there's a conceptual difference between what is an acknowledged "thought experiment" and a "design proposal", I think it's fine to work out ideas in a single heap, and may be the only way to work out some ideas, though it may indeed prove challenging to pull those big ideas back into the main development, but, sometimes you just have to scribble for a while until you see the work that needs to be done.
[21:45] <mdiggory> for which the recommendations, if I recall were... make the stages more explicit for DCAT
[21:46] <tdonohue> mdiggory: Glad to hear positive feedback then on my "Development with Git" page then. I haven't been sure whether what I wrote was "good" or "crazy" :) It was mostly written while I was still digging into Git & learning it myself.
[21:46] <mdiggory> IE For DCAT review, For Committer Review.
[21:46] <hpottinger> speaking of "sketchy" my OR12 notes are very sketchy, but I'll try to make sense of them and add to the meeting notes page
[21:46] <mdiggory> I drankt he cool aid, I'm not yet sure how many others did
[21:47] <tdonohue> were the JIRA workflow suggestions I've made discussed at OR12? Are there changes to that suggestion that need to happen? https://wiki.duraspace.org/display/DSPACE/JIRA+Workflow+Improvements
[21:47] <kompewter> [ JIRA Workflow Improvements - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/JIRA+Workflow+Improvements
[21:47] <tdonohue> (see diagram on that page -- the text is mostly just describing what the diagram shows much better)
[21:48] <hpottinger> yes, I do think there was general agreement on that proposed workflow, tdonohue, no formal vote, though
[21:49] <helix84> i agree with the workflow, too
[21:49] <tdonohue> Ok -- I'll send an email to dspace-commit about the JIRA workflow then, so we can finalize it & make it happen.
[21:49] <hpottinger> I do think we need to add a flag, situational right now, for "has patch, needs pull request"
[21:49] <helix84> and i think i've found nothing blasphemous in your git strategies, either
[21:50] <hpottinger> there are a few issues from the SVN days with patches, just waiting for testing
[21:50] <tdonohue> helix84: good to hear. If you do find something "crazy", feel free to correct it. :)
[21:51] <helix84> i wanted to say that i love the diagram in the git workflow mark mentioned: http://thinkupapp.com/docs/contribute/developers/devfromsource.html
[21:51] <kompewter> [ Develop from Source — ThinkUp 1.0.8.1 documentation ] - http://thinkupapp.com/docs/contribute/developers/devfromsource.html
[21:51] <helix84> (click to enlarge the diagram)
[21:51] <hpottinger> mdiggory used a Presi from ThinkUp as well, very informative, hunting for link
[21:52] <hpottinger> http://prezi.com/itu6w7cvbbb5/github-scm-workflow-for-dspace-contributors/
[21:52] <kompewter> [ Github SCM Workflow for DSpace Contributors by Mark Diggory on Prezi ] - http://prezi.com/itu6w7cvbbb5/github-scm-workflow-for-dspace-contributors/
[21:52] <tdonohue> we can/should just link those into our Git page. I'm all for adding more diagrams around our Git Workflow.
[21:52] <mdiggory> I have a copy of the image with DSpace shamelessly replacing ThinkUp
[21:53] <tdonohue> hopefully ThinkUp won't mind
[21:53] <helix84> adding to OR12 notes
[21:53] <tdonohue> I unfortunately have to head out now...will catch up with any remaining discussion later!
[21:54] <helix84> tdonohue: will you be on IRC tomorrow?
[21:54] <richardrodgers> I have to run also, thanks all
[21:54] * richardrodgers (~richardro@18.189.24.43) Quit (Quit: richardrodgers)
[21:54] <tdonohue> I'm on IRC most every day (as long as I'm working, I'm lurking on IRC). will be on everyday this week
[21:55] <hpottinger> gotta run, too, thanks everyone!
[21:55] <helix84> ok, i'll contact you. bye tim
[21:56] * tdonohue (~tdonohue@c-67-177-108-221.hsd1.il.comcast.net) Quit (Read error: Connection reset by peer)
[21:56] <helix84> mdiggory: are you still here?
[21:57] * hpottinger (~hpottinge@mu-162198.dhcp.missouri.edu) Quit (Quit: Later, taterz!)
[21:58] <mdiggory> yes
[22:01] <helix84> we have some unfinished business :)
[22:02] <helix84> remember DS-1193?
[22:02] <kompewter> [ https://jira.duraspace.org/browse/DS-1193 ] - [#DS-1193] display bitstream read access permissions on item page - DuraSpace JIRA
[22:02] <helix84> link to our IRC conversation is in the last comment
[22:02] <mdiggory> uh oh
[22:03] <mdiggory> did I promise anything?
[22:03] <helix84> that's why i'm bringing it up - i couldn't get you to promise anything :)
[22:04] <helix84> you see, i based by work on status quo and you disagreed with status quo but didn't commit to changing it, so i'm blocked
[22:04] <helix84> s/by/my/
[22:04] <kompewter> helix84 meant to say: you see, i based my work on status quo and you disagreed with status quo but didn't commit to changing it, so i'm blocked
[22:07] <helix84> this week there was someone else on dspace-tech who also made a customization that relies on current behavior
[22:07] <mdiggory> so, we have (1) call to the exposed rightsMD via parameters (2) xslt theming to show results.
[22:08] <mdiggory> and a debate that the exposed rights metadata tot he public might be a problem we should try to fix
[22:08] <helix84> the change you wanted is for access rights not to be accessible in METS so that access to access rights should be restricted
[22:09] <helix84> you wanted access rights accessible as another XML document via another URL
[22:09] <mdiggory> oh yea, I was suggesting to write another transformer that could be access controlled.
[22:09] <helix84> the problem is i have no idea how to do that
[22:12] <mdiggory> heres an example that produces ORE for an Item
[22:12] <mdiggory> https://github.com/DSpace/DSpace/blob/master/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/sitemap.xmap#L447
[22:12] <kompewter> [ DSpace/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/sitemap.xmap at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/sitemap.xmap#L447
[22:12] <mdiggory> https://github.com/DSpace/DSpace/blob/master/dspace-xmlui/dspace-xmlui-api/src/main/java/org/dspace/app/xmlui/cocoon/DSpaceOREGenerator.java
[22:12] <kompewter> [ DSpace/dspace-xmlui/dspace-xmlui-api/src/main/java/org/dspace/app/xmlui/cocoon/DSpaceOREGenerator.java at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-xmlui/dspace-xmlui-api/src/main/java/org/dspace/app/xmlui/cocoon/DSpaceOREGenerator.java
[22:13] <helix84> it sounds like it would be a breeze for you ;)
[22:14] <mdiggory> my time is very limited unfortunately.
[22:15] <mdiggory> So, I would say that unless its something you want to attempt, then we accept your current solution and document this as a future "want"
[22:15] <helix84> that's what I wanted to hear (well, the first thing would be the promise, but this is second best :p )
[22:16] <helix84> I'll attempt it but I don't think I'll succeed
[22:16] <helix84> thanks
[22:20] * barmintor is now known as barmintor_afk
[22:29] <mdiggory> what I would say is that the first pass is good, if we add a hook to block exposing calls to METS from getting outside dspace, thats a good first pass. I think if we enhance DSpace with my recommendations later, its not much to switch your solution to query the separate generator.
[22:29] <mdiggory> if we take that approach, the later work isn't really your problem
[22:30] <mdiggory> and would really just need to be a second jira task
[22:31] <helix84> i'll make sure to file it
[22:43] * lyncode (~DSpace@bl23-51-244.dsl.telepac.pt) Quit (Quit: lyncode)
[23:23] * HugoRibeiro (~Hugo@a89-152-96-76.cpe.netcabo.pt) Quit ()
These logs were automatically created by DuraLogBot on irc.freenode.net using the Java IRC LogBot.