Timestamps are in GMT/BST.
[0:03] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
[6:32] -kornbluth.freenode.net- *** Looking up your hostname...
[6:32] -kornbluth.freenode.net- *** Checking Ident
[6:32] -kornbluth.freenode.net- *** Found your hostname
[6:32] -kornbluth.freenode.net- *** No Ident response
[6:32] * DuraLogBot (~PircBot@ec2-107-22-210-74.compute-1.amazonaws.com) has joined #duraspace
[6:32] * Topic is '[Welcome to DuraSpace - This channel is logged - http://irclogs.duraspace.org/]'
[6:32] * Set by cwilper!ad579d86@gateway/web/freenode/ip.173.87.157.134 on Fri Oct 22 01:19:41 UTC 2010
[12:05] * mhwood (mwood@mhw.ulib.iupui.edu) has joined #duraspace
[12:32] * tdonohue (~tdonohue@c-98-215-0-161.hsd1.il.comcast.net) has joined #duraspace
[12:40] * pbecker (~pbecker@ubwstmapc098.ub.tu-berlin.de) has joined #duraspace
[13:51] * helix84 (~ctenar@195.178.95.132) has joined #duraspace
[14:20] * kdweeks (~Adium@2001:468:c80:a103:e8d4:b5ba:dabb:fa7a) has joined #duraspace
[15:14] * edInCo (~smuxi@seta.coalliance.org) has joined #duraspace
[15:52] * hpottinger (~hpottinge@mu-161174.dhcp.missouri.edu) has joined #duraspace
[15:58] * pbecker (~pbecker@ubwstmapc098.ub.tu-berlin.de) Quit (Quit: Leaving)
[17:53] * peterdietz_ (~peterdiet@server112.longsightgroup.com) has joined #duraspace
[17:53] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) Quit (Ping timeout: 250 seconds)
[17:53] * peterdietz_ is now known as peterdietz
[19:02] * vtkeithg (~vtkeithg@vtkeithg.lib.vt.edu) has joined #duraspace
[19:16] * bram-atmire (~bram@94-225-35-170.access.telenet.be) has joined #duraspace
[19:39] * kdweeks (~Adium@2001:468:c80:a103:e8d4:b5ba:dabb:fa7a) Quit (Quit: Leaving.)
[19:45] * vtkeithg (~vtkeithg@vtkeithg.lib.vt.edu) Quit (Quit: vtkeithg)
[19:52] * KevinVdV (~KevinVdV@d5153D041.access.telenet.be) has joined #duraspace
[20:01] <tdonohue> Hi all, it's 20:00UTC and time for our weekly DSpace Developers Meeting. Rough agenda for today: https://wiki.duraspace.org/display/DSPACE/DevMtg+2014-07-23
[20:01] <kompewter> [ DevMtg 2014-07-23 - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2014-07-23
[20:02] <tdonohue> First off, just wanted to thank everyone for helping to get DSpace 4.2 bug fix release out the door, especially Robin Taylor who "cut" the actual release, etc.
[20:02] * mdiggory (~anonymous@cpe-76-176-128-229.san.res.rr.com) has joined #duraspace
[20:02] * hpottinger waves to Robin as he reads the logs later.
[20:03] <tdonohue> So, now it's pretty much "all hands on deck" for 5.0 release later this year! At this time, there are no official plans for a 4.3 release, unless we have a serious bug or security issue to get out the door ASAP.
[20:03] * bram-atmire cheers!
[20:03] <tdonohue> but fingers crossed that 4.3 won't be needed anytime soon ;)
[20:03] <tdonohue> if at all
[20:04] * pbecker (55d42c85@gateway/web/freenode/ip.85.212.44.133) has joined #duraspace
[20:04] <pbecker> Hi
[20:04] <tdonohue> So, that's basically it on the 4.2 front. The rest of the meeting I wanted to start some planning and/or discussion around 5.0
[20:05] <hpottinger> pbecker is here just in time to volunteer for the 5.0 RT
[20:05] <tdonohue> And, hpottinger is reminding us that we are still looking for some 5.0 Release Team (RT) members, if anyone is interested. You can join the esteemed hpottinger & peterdietz!
[20:05] <pbecker> Hpottinger: nice try, but I'm afraid I cannot be part of the rt this year. Sorry.
[20:07] <pbecker> I will concentrate on my major contribution dspace-rdf, if it'll get into 5.0.
[20:07] <hpottinger> we'll be here if you change your mind... any other takers?
[20:07] <tdonohue> I think I've neglected to mention this to almost everyone...the 5.0 release is going to coincide with the birth of my second daughter (due in late Nov.), so I'm going to be likely out-of-touch right around the actual 5.0 release time. So, anyone who can find time to chip in will be very much appreciated.
[20:08] <peterdietz> congrats!
[20:09] <tdonohue> Obviously, I'll be chipping as much as I can before the final release....but, I'll be sleep deprived and/or offline much of late Nov and Dec
[20:09] <bram-atmire> congratulations Tim!
[20:09] <pbecker> tdonohue: congratulations!
[20:09] * kdweeks (~Adium@2001:468:c80:a103:25d3:c9f4:82cd:58d8) has joined #duraspace
[20:09] <tdonohue> thanks all...we are very excited and happy obviously!
[20:10] <mhwood> Best wishes to all. Let us know what we can do here.
[20:10] <hpottinger> grats, Tim!
[20:10] <peterdietz> Things I'm thinking of for 5 are: batch UI import, rest management tools, maybe even configurable i18n from the UI... Basically, things that make it easier to manage a DSpace instance
[20:10] <tdonohue> Back to 5.0 timelines though, as of right now, reminder what the tentative schedule looks like: https://wiki.duraspace.org/display/DSPACE/DSpace+Release+5.0+Status#DSpaceRelease5.0Status-TimelineandProcessing
[20:10] * hpottinger swoons.
[20:11] <kompewter> [ DSpace Release 5.0 Status - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DSpace+Release+5.0+Status#DSpaceRelease5.0Status-TimelineandProcessing
[20:11] <tdonohue> peterdietz ++ (any and/or all of those would be awesome)
[20:12] <bram-atmire> peterdietz if you’re doing the i18n work check out https://jira.duraspace.org/browse/DS-1583
[20:12] <kompewter> [ [DS-1583] Adopt interface translating by Translatewiki.net - DuraSpace JIRA ] - https://jira.duraspace.org/browse/DS-1583
[20:12] <kompewter> [ https://jira.duraspace.org/browse/DS-1583 ] - [DS-1583] Adopt interface translating by Translatewiki.net - DuraSpace JIRA
[20:12] <tdonohue> Essentially, though, based on the timeline, we are just a little over *2 months* away from the proposed "Deadline for Feature Pull Requests". Time is going quickly
[20:12] <bram-atmire> and the PR where email messages are broken up into the catalog
[20:13] * pbecker (55d42c85@gateway/web/freenode/ip.85.212.44.133) Quit (Quit: Page closed)
[20:14] * pbecker2 (55d42c85@gateway/web/freenode/ip.85.212.44.133) has joined #duraspace
[20:15] <tdonohue> At this point though, I'm willing to open up to any thoughts/brainstorms/works in progress anyone wants to talk about for 5.0... or even just questions in general
[20:15] <pbecker2> I would like to talk about Linked Data Support
[20:15] <tdonohue> (I don't have any true agenda here on 5.0...just trying to get it into the forefront of everyone's minds)
[20:15] <mhwood> There's a question just went out on -devel about using HTML5 features.
[20:16] <hpottinger> we have three people from atmire here today
[20:16] <mdiggory> ;-)
[20:16] <bram-atmire> ;)
[20:16] <tdonohue> RE: HTML5 -- I'm +1 moving towards HTML5. But, I would be curious to hear if anyone else has major concerns, etc.
[20:16] <tdonohue> yes, and hello there @mire office(s)
[20:16] <mdiggory> <-- admittedly lurking
[20:17] <mhwood> pbecker2 got his topic in first. Linked Data, anyone?
[20:17] <mdiggory> ok, maybe not lurking…
[20:17] <pbecker2> we can go on with html5 and come back to linked data later or next week. ;-)
[20:18] <mhwood> OK, I just wonder if HTML5 support is pervasive enough to depend on it.
[20:18] * pbecker (~pbecker@55d42c85.access.ecotel.net) has joined #duraspace
[20:18] * pbecker2 (55d42c85@gateway/web/freenode/ip.85.212.44.133) Quit (Client Quit)
[20:19] <tdonohue> pbecker: might as well quickly plug your work though..so people can start reviewing it (even offline). Here is pbecker's Linked Data work, for folks to start to review: DS-2061
[20:19] <kompewter> [ https://jira.duraspace.org/browse/DS-2061 ] - [DS-2061] Linked (Open) Data support for DSpace - DuraSpace JIRA
[20:19] <peterdietz> i would say go for html5: http://caniuse.com/#cats=HTML5
[20:19] <kompewter> [ Can I use... Support tables for HTML5, CSS3, etc ] - http://caniuse.com/#cats=HTML5
[20:19] <pbecker> mhwood, tdonohue: thanks.
[20:20] <pbecker> perhaps as part of HTML5: DS-1994.
[20:20] <kompewter> [ https://jira.duraspace.org/browse/DS-1994 ] - [DS-1994] Use HTML5 to upload files in Submission Process of JSPUI - DuraSpace JIRA
[20:20] <tdonohue> pbecker: your 1994 ticket actually prompted this HTML5 discussion. We discussed it during the "JIRA backlog review" and wanted to try and get it in
[20:21] <pbecker> cool. I did not head time to read my mails. Just came in just in time for the developer meeting.
[20:21] <mdiggory> pbecker: can you comment on SPARQL support in your branch?
[20:21] <pbecker> mdiggory: of course. What do you mean exactly?
[20:22] <mdiggory> --> To do so it should provide the converted data as serialization of RDF and over a SPARQL endpoint.
[20:22] <tdonohue> pbecker meet mdiggory (if you haven't already). mdiggory is big into Linked Data / RDF, etc ;)
[20:22] <mdiggory> lol
[20:22] <pbecker> :-)
[20:22] <mdiggory> thanks tdonohue
[20:22] <pbecker> I use SPARQL 1.1 Graph Store HTTP Protocol to connect the Triple store to the repositoriy.
[20:22] <tdonohue> just wanted to provide some background :)
[20:23] <pbecker> The Triple Store itself provides all converted data over a public readable SPARQL endpoint (if configured correctly).
[20:24] <mdiggory> I see support for consumers to produce RDF and Content Negotiation in your work are you also providing a SPARQL endpoint?
[20:25] <pbecker> I convert the repository content into rdf and stores the rdf in a triple store. The triple store provides a sparql endpoint.
[20:25] <mdiggory> what triplestore technology are you using at this point?
[20:26] <hpottinger> DSPR#568
[20:26] <kompewter> [ https://github.com/DSpace/DSpace/pull/568 ] - DS-2061: Linked (Open) Data support for DSpace by pnbecker
[20:26] <pbecker> I'm using Apache Jena Fuseki, but you can use every Triple Store that supports Sparql 1.1 Graph Store HTTP Protocol
[20:27] <mdiggory> how do you find the write performance on Fuseki?
[20:27] <pbecker> I don't care mutch about the write performance, as I suppose that data in repositories is much more read then it is written or changed.
[20:28] <pbecker> And cause I'm using Sparql to connect the Triple Store I can change the used Triple Store easily
[20:28] <pbecker> You can really use the triple store you like, as long as it support Sparql 1.1
[20:28] <mdiggory> the write performance is important from a Consumer standpoint when new content is being imported
[20:29] <pbecker> I had no problbems in my tests.
[20:29] <pbecker> but I have not done any speed tests.
[20:29] <mdiggory> I discuss because we have a project with WHOI that is similar.
[20:29] <pbecker> I can mail you access to my test repository tomorrow.
[20:30] <mdiggory> We leverage the OpenRDF Sesame API for serialization and are currently providing a Sesame repository for SPARQL support
[20:30] <tdonohue> WHOI = http://www.whoi.edu/ ?
[20:30] <kompewter> [ Home : Woods Hole Oceanographic Institution ] - http://www.whoi.edu/
[20:30] <mdiggory> tdonohue: thanks again
[20:31] <mdiggory> of course, preformance is a challenge with non-native triplestores (java) as the number of triples increases
[20:32] <pbecker> I'm not using it in a productive repository right now, cause it is not completly finished (and there is a problem in DSpace that prevents us from using DOIs in LD). I only have a quite small testrepository.
[20:32] <pbecker> But as I set before, if there is a Problem with fuseki, we can change the Triple Store really easily.
[20:33] <pbecker> s/set/said
[20:33] <kompewter> pbecker meant to say: But as I said before, if there is a Problem with fuseki, we can change the Triple Store really easily.
[20:33] <mdiggory> yes, similarly with our work.
[20:34] <pbecker> mdiggory: I would be glad, if you would take a look on DS-2061.
[20:34] <kompewter> [ https://jira.duraspace.org/browse/DS-2061 ] - [DS-2061] Linked (Open) Data support for DSpace - DuraSpace JIRA
[20:34] <tdonohue> So, wondering if there's an opportunity for @mire / mdiggory to collaborate (or give more direct feedback based on experiences) with pbecker on this work. Not trying to put anyone on the spot for anything immediate, just sounds like common interests emerging which could improve on one another
[20:34] <mdiggory> there is some great overlap, we have written consumers and a serialization "framework"
[20:35] <pbecker> mdiggory: Can I take a look on that? Is there some information available about it?
[20:35] <mdiggory> our challenge was that we have alot of "subject centric" RDF that needs to be produced for WHOI that is specific to oceanographic research. So the framework is a customizable/pluggable serialization layer
[20:36] <mdiggory> pbecker: we can work out a means to get you a look
[20:37] <pbecker> just to be sure, to understand you correctly: a serialization layer to serialize content/metadata stored in DSpace as RDF?
[20:37] <tdonohue> sounds like a good idea :)
[20:37] <mdiggory> pbecker: yes
[20:38] <mdiggory> http://semanticweb.com/easing-way-linked-open-data-geosciences-domain_b42955
[20:38] <kompewter> [ Easing The Way To Linked Open Data In The Geosciences Domain - Semanticweb.com ] - http://semanticweb.com/easing-way-linked-open-data-geosciences-domain_b42955
[20:38] <pbecker> funny, so our code really might overlap.
[20:39] <mdiggory> http://www.oceanlink.org/
[20:39] <kompewter> [ OceanLink Portal ] - http://www.oceanlink.org/
[20:41] <tdonohue> So...it sounds like a discussion is definitely warranted. Any other last thoughts on LinkedData (from anyone)? Honestly, I just look forward to hearing what comes out of this
[20:42] <tdonohue> Back to HTML5 thread...any other thoughts there that someone wants to add? As mentioned, there's also a discussion thread started on dspace-devel by mhwood
[20:43] <hpottinger> Or, you know, if you're suddenly overcome with the desire to volunteer for the 5.0 RT...
[20:44] <mhwood> Multiple file selection seems like the main thing wanted, and it seems to be well supported on everything but backlevel IE and phones.
[20:44] <tdonohue> +1 mhwood
[20:44] <mhwood> I'm trying not to imagine people submitting hour-long videos from phones.
[20:44] <tdonohue> Ok, since those discussions seem to have mostly wrapped up, I'll bring up DS-2072 (which was just realized/discovered by pbecker, peterdietz & I today)
[20:45] <kompewter> [ https://jira.duraspace.org/browse/DS-2072 ] - [DS-2072] XMLUI always commits any uncommitted changes while JSPUI always discards any uncommitted changes - DuraSpace JIRA
[20:45] <tdonohue> and it seems to be related to an older ticket, DS-383
[20:45] <kompewter> [ https://jira.duraspace.org/browse/DS-383 ] - [DS-383] Auto commit functionality not working correctly in XMLUI - DuraSpace JIRA
[20:45] <mhwood> "Backlevel" here means <v10 which is actually not all that old, given that Microsoft only recently jettisoned IE6.
[20:46] <hpottinger> re 2072: what's the "right answer"?
[20:46] <tdonohue> 2072 is a bit worrisome, as we have two UIs which do entirely different things with uncommitted changes...and it causes very odd bugs to pop up along the way (like DS-2012, which only affects JSPUI cause it gets "auto-committed" by XMLUI)
[20:47] <kompewter> [ https://jira.duraspace.org/browse/DS-2012 ] - [DS-2012] JSPUI does not write column last_active in database as context is not commited after login - DuraSpace JIRA
[20:47] <mhwood> I would suppose that there is code in each which depends on that final disposition. We'd have to comb through them carefully, if we want to change either.
[20:47] <tdonohue> hpottinger: that's essentially what I'm asking all of you (and I also asked on dspace-devel). The "right answer" to me is that we need both UIs to act the same way...but I'm open to suggestions
[20:48] <mhwood> I would like them to work the same, for sanity's sake, but it may not be simple.
[20:48] <tdonohue> mhwood: yep, I agree.
[20:48] <tdonohue> which is also why I figured I'd bring it up now...as we are very early in the 5.0 timeline still
[20:50] <tdonohue> Perhaps it's too much info at once here :) I did also start a thread on dspace-devel for comments/suggestions...it's just something I hope we can work to try and resolve (one way or the other), preferably for 5.0
[20:50] <hpottinger> so, if we make JSPUI commit uncommitted changes, we fix 2012 and 2072... and maybe break something else?
[20:51] <mhwood> So, if we were staring at blank editor windows, ready to code up XMLUI and JSPUI from scratch, which way would we choose: finally commit or finally abort?
[20:52] <pbecker> mhwood: that is a really good question.
[20:52] <tdonohue> mhwood: yes, that's the question. Should our "finally()" cleanup by auto-committing any changes from the last request (which didn't result in errors)....or should it just throw them out (abort). The former is what XMLUI does, the latter JSPUI
[20:53] <peterdietz> If I were staring at a blank editor window, I'd make Rails UI, that uses REST... I'm not sure where the context committing comes into play at that point though
[20:54] <tdonohue> hpottinger: having the JSPUI do an auto-commit likely won't "break" anything...just more a question of whether that seems "OK". Having the XMLUI do an abort() however likely will break (some or many) things (hard to know for certain).
[20:55] <peterdietz> For this issue, I'd be cautious about not creating too many new issues, and just adding a context.commit in jspui, shortly after the login...
[20:55] <hpottinger> if XMLUI does an abort, we'll have to explicitely commit certain things after login, (DS-2012)
[20:55] <kompewter> [ https://jira.duraspace.org/browse/DS-2012 ] - [DS-2012] JSPUI does not write column last_active in database as context is not commited after login - DuraSpace JIRA
[20:56] <pbecker> I would be afraid to commit anything without knowing what gets commited. so if I would start in a blank editor, I would abort the context, if it was not commited before.
[20:58] <mhwood> Well, my question supposes that we are going to design the code to be compatible with one or the other. Is one a better starting point than the other? Which one do we *wish we had*, so we know what to look for, and roughly where, if we want to fix this?
[20:58] <peterdietz> i.e. if you have an action, like Edit Item. You should commit that changed value right away. And not rely on mysterious end-of-the-line "garbage collector"
[20:59] <tdonohue> See, for me, I actually kinda like what XMLUI does...but it could be deemed "unsafe" practices (for the reasons pbecker states, which I can completely understand). XMLUI says, if an error happened, I'm aborting...but as long as things didn't error out, lets ensure it all got committed.
[20:59] <bram-atmire> peterdietz +
[20:59] <mhwood> I was going to say that's a third choice, but having no catch-all means that the catch-all is abort, when the Context is destroyed, yes?
[20:59] <tdonohue> so, peterdietz, pbecker & bram-atmire all seem to like "the JSPUI way"
[20:59] <hpottinger> see, when tdonohue puts it that way, it seems benign...
[21:00] <pbecker> I prefere "the JSPUI way", but I can imagine how much it would hurt to get this into XMLUI without knowing XMLUI well.
[21:01] <mhwood> Another way of looking at this is that some of us don't have a good grasp of the transaction boundaries.
[21:03] <hpottinger> perhaps the "right way" is the JSPUI way, but I worry about the risks of not finding and fixing all the dependencies on the old behavior
[21:03] <tdonohue> yea, fixing the XMLUI could be a bit of an ordeal...but, if we make a decision that "the JSPUI way" is more correct, then we should start to investigate fixing the XMLUI to see where it misses "commit()" calls...maybe there's only a few...we won't know until we dig deeper. I was more trying to gage whether we need to dig deeper or not.
[21:04] <tdonohue> and it sounds like we have more folks leaning towards "the JSPUI way"...which answers the question. We need to dig into the XMLUI to see how hard it would be to "fix it" to act like the JSPUI in this scenario.
[21:04] <tdonohue> (and whether or not that's achievable for 5.0...or if it just remains a "known issue")
[21:04] <mhwood> I think I agree. If you don't know what state the Context is in, abort it before you go.
[21:05] <hpottinger> so the fix for 2012 is to add a context commit somewhere after login, and we'll have to do it for XMLUI, too
[21:05] <mhwood> Don't save unknown stuff.
[21:05] <mhwood> Commit goes in dspace-api and we make sure that all UIs are okay with this.
[21:05] <KevinVdV> I agree with mhwood, don’t save if you don’t know what you are saving !
[21:06] <KevinVdV> => Needs to run now though, until next time !
[21:06] * KevinVdV (~KevinVdV@d5153D041.access.telenet.be) Quit (Quit: KevinVdV)
[21:06] <mhwood> I need to go, too.
[21:06] <tdonohue> hpottinger: I still personally think the fix for 2012 is to do a *commit* immediately after the "eperson.update()" call. There should be nothing else on the Context at that point, as you just authenticated. I.e. add a commit here: https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/AuthenticationManager.java#L174
[21:06] <kompewter> [ DSpace/dspace-api/src/main/java/org/dspace/authenticate/AuthenticationManager.java at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/AuthenticationManager.java#L174
[21:06] <peterdietz> I'm quite lazy, I might prefer to leave this as a known gotcha. I'm not looking forward to testing all 250 XMLUI servlets and sub-servlets to determine that we've caught everything.
[21:06] <hpottinger> but, also, *save* if you need to save
[21:07] <pbecker> tdonohue: I would not add a commit in dspace-api.
[21:07] <hpottinger> it does seem like a high risk for a theoretical gain
[21:07] <pbecker> I would never expect, that a login method commits a hole transaction.
[21:08] <tdonohue> Notes that yes, we are "over time". So, we'll just wrap things up for today. I'll write up the decision into DS-2072...we can continue to figure out the "best fix" for Ds-2012
[21:08] <kompewter> [ https://jira.duraspace.org/browse/DS-2072 ] - [DS-2072] XMLUI always commits any uncommitted changes while JSPUI always discards any uncommitted changes - DuraSpace JIRA
[21:08] <bram-atmire> gtg, good night everyone
[21:08] <tdonohue> And, I can stick around for a bit as necessary...if folks want to continue...but feel free to drop off if you need to! No more official topics today
[21:09] * mhwood (mwood@mhw.ulib.iupui.edu) has left #duraspace
[21:09] * bram-atmire (~bram@94-225-35-170.access.telenet.be) Quit (Quit: bram-atmire)
[21:09] <pbecker> tdonohoue: I have some minutes left. If you have too, I would like to talk about ds-2012.
[21:09] <kompewter> [ https://jira.duraspace.org/browse/ds-2012 ] - ('Unexpected error:', <type 'exceptions.AttributeError'>)
[21:10] <tdonohue> sure, we can talk about 2012 a bit more...
[21:11] <pbecker> I think the question left is where to commit. isn't it?
[21:11] <tdonohue> yes, I think so. I just don't like having to commit this one tiny change all over. Seems a tad duplicative to me.
[21:12] <pbecker> Is there any other example in dspace-api where a commit is done?
[21:12] <tdonohue> And, looking at the AuthenticationManager, I don't understand *any* scenario where adding a commit() there would hurt....there should be nothing else waiting to be committed, as you just logged in
[21:12] <pbecker> just askin, cause you know the code better then me.
[21:13] <pbecker> I'm not thinking if it really hurts, I'm thinking what I would expect from the API.
[21:13] <pbecker> I'm just afraid that it might hurt one day in the future or that we might ask us once, why we added a commit there...
[21:13] <pbecker> I would never expect a commit there.
[21:14] <tdonohue> Here's an example of a commit() inside another Authentication method: https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/LDAPAuthentication.java#L278
[21:14] <kompewter> [ DSpace/dspace-api/src/main/java/org/dspace/authenticate/LDAPAuthentication.java at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/LDAPAuthentication.java#L278
[21:14] <pbecker> and perhaps this is a little bit the same as ds2072. are you optimistic that there is nothing to commit or are you expection that there might be somtihng.....
[21:14] <tdonohue> (and it's immediately after an eperson.update(), just like this scenario)
[21:15] <tdonohue> and another, this time in Shib: https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/ShibAuthentication.java#L722
[21:15] <kompewter> [ DSpace/dspace-api/src/main/java/org/dspace/authenticate/ShibAuthentication.java at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/ShibAuthentication.java#L722
[21:16] <tdonohue> and another in X509 auth: https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/X509Authentication.java#L622
[21:16] <kompewter> [ DSpace/dspace-api/src/main/java/org/dspace/authenticate/X509Authentication.java at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/X509Authentication.java#L622
[21:17] <tdonohue> none in PasswordAuthentication class though...which is what we are testing this against
[21:18] <pbecker> I tested ds-2012 in shib as well.
[21:18] <kompewter> [ https://jira.duraspace.org/browse/ds-2012 ] - ('Unexpected error:', <type 'exceptions.AttributeError'>)
[21:18] <tdonohue> But, to answer your question...yes, we actually do have commit() calls in the dspace-api. A search brought up 149 results for me (searched just dspace-api in NetBeans)
[21:18] <pbecker> interessting.
[21:19] <tdonohue> although some of those are *Solr* commits...not Context.commit() now that I look more closely
[21:20] <tdonohue> The reason here is that our API is a bit of a mishmash...it's not just a simple API layer...it also does contain some Business Logic which is shared across both UIs (as we don't have a separate "Business Logic" layer to put that shared code into...other than the new REST API, which currently isn't used by either UI)
[21:21] <pbecker> I know.
[21:21] <pbecker> Do you see my point, why I would never expect a commit there?
[21:21] <tdonohue> So, If, for example the UIs both used REST API...then it'd make more sense to have more of the "commit" calls come in REST API, rather than in the dspace-api
[21:21] <pbecker> I'm not totaly against commiting their, Im just curious if this is really a good way.
[21:23] <tdonohue> pbecker: I can understand your concerns and confusion here
[21:24] <tdonohue> I guess the other way of looking at this... do most of the JSPUI servlets call context.commit() or complete()? If so, then maybe that is the "best practice" for JSPUI afterall.... (I admit I mostly do XMLUI these days)
[21:26] <tdonohue> i.e. part of this may be an "XMLUI way of doing things" vs "JSPUI way of doing things" question
[21:26] <pbecker> I'll take a look on that tomorrow.
[21:26] <pbecker> good point.
[21:26] <pbecker> have to leave now.
[21:26] <pbecker> was a good developer meeting today. :-)
[21:27] <tdonohue> So, if it ends up being the "best practices" for JSPUI, then I'm not against 2012
[21:27] <tdonohue> thanks! have a good evening
[21:27] <pbecker> I'll take a look on other jspui servlets tomorrow.
[21:27] <pbecker> thanks, you too.
[21:27] <pbecker> by
[21:27] <pbecker> e
[21:27] * pbecker (~pbecker@55d42c85.access.ecotel.net) Quit (Quit: pbecker)
[21:29] <mdiggory> https://jira.duraspace.org/browse/DS-383 the problem with peterdietz statement about autocommit is that forcing commits into solr causes bottlenecks to occur when they are frequent and take long periods to complete…
[21:29] <kompewter> [ [DS-383] Auto commit functionality not working correctly in XMLUI - DuraSpace JIRA ] - https://jira.duraspace.org/browse/DS-383
[21:29] <kompewter> [ https://jira.duraspace.org/browse/DS-383 ] - [DS-383] Auto commit functionality not working correctly in XMLUI - DuraSpace JIRA
[21:30] <mdiggory> peterdietz: i.e. if you have an action, like Edit Item. You should commit that changed value right away. And not rely on mysterious end-of-the-line "garbage collector"
[21:30] <mdiggory> [1:59pm]
[21:32] <mdiggory> if you are importing alot of items and each one gets a db commit to assure that you can keep the previous ones should an error arise… you are issuing new solr writes in each case that a Context commit happens. repeatedly commiting solr in this situation forces it to reload its searchers and in-memory caches inefficently
[21:33] <mdiggory> autocommiting takes this responsibility away from the code and allow solr to be optimized in this situation
[21:34] <mdiggory> as we get into more situations where mutliple items are updated at once, the more important it will be to not force a commit after each item is reindexed
[21:35] <peterdietz> I think I'm talking about database/context commit. After you do something in the UI to change a single item. It should write that to the Database. Won't an eventlistener get notified to add a change to SOLR's queue
[21:36] <peterdietz> If you had a batch changer, you'll probably not commit after each singular change. But wrap the series of changes into a transaction that you commit at the end.
[21:37] <peterdietz> I've noticed that newly submitted items (in DSpace4) don't show up in solr right away, but maybe after 10 minutes they appear. If I do /dspace/bin/dspace index-discovery they show up right after
[21:37] <peterdietz> I think after a submission might be a good time for a SOLR.commit
[21:38] <mdiggory> it depends on the case, if you want to be able to rollback the whole set of changes on failure of one, yes
[21:38] <mdiggory> however, there are cases where you want to incrementally commit to the db and the have some "marker" at the failure point (example ItemImport)
[21:41] <mdiggory> I see what your saying now about referring to db commiting...
[21:42] <mdiggory> apparently I was quite active on this issue ;-)
[21:43] <mdiggory> https://jira.duraspace.org/browse/DS-2072
[21:43] <kompewter> [ [DS-2072] XMLUI always commits any uncommitted changes while JSPUI always discards any uncommitted changes - DuraSpace JIRA ] - https://jira.duraspace.org/browse/DS-2072
[21:43] <kompewter> [ https://jira.duraspace.org/browse/DS-2072 ] - [DS-2072] XMLUI always commits any uncommitted changes while JSPUI always discards any uncommitted changes - DuraSpace JIRA
[21:43] <mdiggory> This is why we need a business tier.
[21:45] <mdiggory> I beleive that in XMLUI the autocommit functionality is designed so that you don't commit in midstream processing as many different changes may be enacted by a request in the Cocoon pipeline (i.e. multiple actions)
[21:46] <mdiggory> the goal is to allow developers to insert new actions into the pipleine and beable to allow additional changes to be added within that commit window
[21:47] <mdiggory> I beleive it is an intentional design that could probibly be confirmed by checking with Scott Phillips
[21:52] <tdonohue> mdiggory, could you please copy & paste that into 2072 comments? I have to run, but that does actually sound completely plausible
[21:52] <mdiggory> I just beat you to the punch :-)
[21:53] <tdonohue> awesome! Just saw that, thanks!
[21:53] <peterdietz> sounds reasonable. As long as none of the pipelines have to abort the context (rollback/zero-out the changes), then the finally says to context.commit
[21:53] <tdonohue> and if that is the case...this may be a developer documentation issue...letting everyone know WHY this works how it does, etc
[21:53] <tdonohue> gotta go now...later
[21:53] * tdonohue (~tdonohue@c-98-215-0-161.hsd1.il.comcast.net) has left #duraspace
[22:11] * peterdietz (~peterdiet@server112.longsightgroup.com) Quit (Quit: peterdietz)
[22:12] * kshepherd slopes in late and scrolls back, as usual
[22:14] * edInCo (~smuxi@seta.coalliance.org) Quit (Remote host closed the connection)
[22:14] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
[22:19] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) Quit (Quit: peterdietz)
[22:37] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
[22:44] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) Quit (Quit: peterdietz)
[22:49] * hpottinger (~hpottinge@mu-161174.dhcp.missouri.edu) has left #duraspace
[22:49] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
[23:04] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) Quit (Quit: peterdietz)
[23:23] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
[23:42] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) Quit (Quit: peterdietz)
[23:44] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
[23:47] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) Quit (Client Quit)
[23:57] * kdweeks (~Adium@2001:468:c80:a103:25d3:c9f4:82cd:58d8) Quit (Quit: Leaving.)
[23:57] * peterdietz (~peterdiet@162-231-20-67.lightspeed.clmboh.sbcglobal.net) has joined #duraspace
These logs were automatically created by DuraLogBot on irc.freenode.net using the Java IRC LogBot.