Timestamps are in GMT/BST.
[6:45] -weber.freenode.net- *** Looking up your hostname...
[6:45] -weber.freenode.net- *** Checking Ident
[6:45] -weber.freenode.net- *** Found your hostname
[6:45] -weber.freenode.net- *** No Ident response
[6:45] * DuraLogBot (~PircBot@webster.duraspace.org) has joined #duraspace
[6:45] * Topic is 'Welcome to DuraSpace IRC. This channel is used for formal meetings and is logged - http://irclogs.duraspace.org/'
[6:45] * Set by tdonohue on Thu Sep 15 17:49:38 UTC 2016
[12:23] * mhwood (~mhwood@mhw.ulib.iupui.edu) has joined #duraspace
[12:24] * tdonohue (~tdonohue@dspace/tdonohue) has joined #duraspace
[13:52] <DSpaceSlackBot1> <tdonohue> Reminder that we have our weekly DSpace DevMtg today at 15UTC (11am ET, a little over an hour from now) here. Agenda was just posted to: https://wiki.duraspace.org/display/DSPACE/DevMtg+2017-09-06
[15:00] <DSpaceSlackBot1> <tdonohue> Hi all @here. It's DSpace DevMtg time. Our agenda is posted above :point_up:
[15:01] <DSpaceSlackBot1> <terrywbrady> I have a conflicting meeting today. I will join in if it ends early.
[15:01] <DSpaceSlackBot1> <sulfrian> Hi
[15:01] <DSpaceSlackBot1> <tdonohue> Looks like more folks are around today (at least lurking in Slack) which is good to see. :slightly_smiling_face: Welcome all.
[15:01] <DSpaceSlackBot1> <pbecker> Hello everyone.
[15:02] <DSpaceSlackBot1> <mwood> Hello.
[15:02] <DSpaceSlackBot1> <tdonohue> The main topic for today's meeting is the 6.2 release, and (hopefully) trying to nail down a release date for 6.2
[15:02] <DSpaceSlackBot1> <pbecker> I just added a reference to DS-3681 to the meeting schedule. If possible, I would like to see that going into 6.2.
[15:03] <DSpaceSlackBot1> <tdonohue> As of a week ago, demo.dspace.org has been running 6.2. But, I'll admit, I haven't heard any feedback (good or bad) on whether folks have been testing demo
[15:03] <DSpaceSlackBot1> <pbecker> But I should mention it is not a reason to delay 6.2.
[15:04] <DSpaceSlackBot1> <pbecker> @sulfrian created a PR against it (thank you!). I reviewed the PR it looks fine to me. We would need a second +1, a vote from mhwood as release committer and someone to test it.
[15:05] <DSpaceSlackBot1> <tdonohue> @pbecker and @sulfrian : To be honest, I think 6.2 has been high priority for some time (cause of known stability/scalability issues in 6.x). I worry that 3681 needs more eyes/testing, and I'm hesitant to throw in a larger refactor here (as it could have unforeseen side effects or accidentally introduce new bugs). That isn't to say it shouldn't get into 6.x soonish, but I'm very hesitant to push more into
[15:05] <DSpaceSlackBot1> <tdonohue> But, that's just my opinion here, and would like to hear from others
[15:07] <DSpaceSlackBot1> <mwood> I think that 6_x is going to have quite a few releases, because we seem to uncover more Hibernate/cache problems whenever we fix one. We need to get fixes out frequently until we seem to have found them all.
[15:08] <DSpaceSlackBot1> <pbecker> It is a direct follow-up of some hibernate problems and touches only the authority-indexer, so the side-effects should be limited. But If you all think it needs more tester, I would understand nevertheless.
[15:09] <DSpaceSlackBot1> <tdonohue> well said @mwood. Yes, I'd rather we do smaller, speedier releases to 6.x. I wish 6.2 had been out a few weeks back (as there are significant fixes in there that folks need in production) and I don't want to delay it any further at this point.
[15:09] <DSpaceSlackBot1> <pbecker> I'm a bit afraid that people who are using the authority index (ORCID) will either need a lot of memory to rebuild that index or will run into problems.
[15:09] <DSpaceSlackBot1> <pbecker> I agree that it shouldn't delay 6.2
[15:10] <DSpaceSlackBot1> <tdonohue> So, let's set 3681 aside for now, and talk 6.2 first then
[15:11] <DSpaceSlackBot1> <tdonohue> I haven't heard any specific news/feedback on the pre-6.2 out on demo.dspace.org. But, also demo hasn't crashed in the last week (that I've seen), so that's something. Do we want to do some of our own quick testing of demo still? Or do we feel 6.2 is "ready to go"?
[15:12] <DSpaceSlackBot1> <pbecker> I'm happy we reverted the changes in the context class. I think that made it much easier to test. Personally I think it is ready.
[15:13] <DSpaceSlackBot1> <tdonohue> Yes, thanks for that update @pbecker and your additional testing that discovered the Context changes were dangerous
[15:14] <DSpaceSlackBot1> <tdonohue> So, if we feel 6.2 is "ready"....then all we really need is someone to volunteer to cut the release (@mwood, are you still interested?) and a date to cut it on/by. I do recall last week we tentatively said Thurs/Fri of this week as a release date
[15:14] <DSpaceSlackBot1> <hpottinger> I've been running what is essentially a preview of 6.2 in staging for a couple of weeks, no complaints
[15:14] <DSpaceSlackBot1> <mwood> I still have 6.2 release on my calendar.
[15:15] <DSpaceSlackBot1> <pbecker> @hpottinger so do we in our test system (production is waiting for a bigger migration)
[15:15] <DSpaceSlackBot1> <tdonohue> @mwood : Remind me what date we said? Is it on your calendar for tomorrow or Fri? :slightly_smiling_face:
[15:15] <DSpaceSlackBot1> <mwood> Tentative Thursday, definitely by Friday.
[15:15] <DSpaceSlackBot1> <hpottinger> so... honest question, how many times do people actually run the reindex for authority?
[15:16] <DSpaceSlackBot1> <tdonohue> @hpottinger : I don't know the answer there...but I don't want to accidentally break it to the point that we have to rush out an immediate 6.3 the same week ;)
[15:16] <DSpaceSlackBot1> <pbecker> @hpottinger at least when they start using it and it was introduced with DSpace 6.
[15:16] <DSpaceSlackBot1> <pbecker> ORCID is a point that currently gets quite some attention
[15:17] <DSpaceSlackBot1> <hpottinger> ```# run the index-authority script once a day at 12:45 to ensure the Solr Authority cache is up to date #45 0 * * * $DSPACE/bin/dspace index-authority > /dev/null ```
[15:17] <DSpaceSlackBot1> <hpottinger> commented out
[15:17] <DSpaceSlackBot1> <tdonohue> As far as I'm concerned, an untested PR that does code refactoring is something we don't rush into a release. It needs testing & analysis, and if 6.2 is "ready to go", we should just release 6.2 and get this refactor in 6.x for a 6.3 in the near future
[15:17] <DSpaceSlackBot1> <pbecker> Is someone @here who would volunteer to test it?
[15:18] <DSpaceSlackBot1> <pbecker> I just haven't tested it as we are not using ORCID.
[15:18] <DSpaceSlackBot1> <hpottinger> I agree, 6.2 is OK as-is, and I can test it, though it's not clear at me what benefit one would even achieve by re-running the index script
[15:19] <DSpaceSlackBot1> <tdonohue> I honestly think that if we are saying we shouldn't delay 6.2 for 3681 *AND* we feel 6.2 is ready, THEN that means we release 6.2 tomorrow (or whenever @mwood is ready) and work on getting 3681 tested & into 6.x just after that release
[15:19] <DSpaceSlackBot1> <pbecker> @hpottinger the index script is needed only for the initial indexing. If you install a empty repository and switch it on right away, you shouldn't need it at all.
[15:20] <DSpaceSlackBot1> <pbecker> @tdonohue fair enough. @mwood go ahead!
[15:20] <DSpaceSlackBot1> <hpottinger> OK, that's the cause of my confusion: I've been using ORCID on our staging repository since we stood it up
[15:21] <DSpaceSlackBot1> <mwood> And if you are upgrading to 6_x and beginning to use the authority feature, then indexing should work if given enough memory, yes?
[15:21] <DSpaceSlackBot1> <pbecker> Depends on the size of your repo.
[15:21] <DSpaceSlackBot1> <pbecker> It currently needs to be able to cache *all* your items.
[15:21] <DSpaceSlackBot1> <hpottinger> Fun thing about Java8/Tomcat 8 is, you can finally just feed your box more memory and it "does the right thing"
[15:22] <DSpaceSlackBot1> <pbecker> Maybe we should mention it in the release notes or in our documentation as known bug? pointing to the PR and ask for testers?
[15:22] <DSpaceSlackBot1> <tdonohue> If we get 3681 into 6.x shortly, then sites that need it immediately can just cherry-pick it immediately
[15:22] <DSpaceSlackBot1> <mwood> We do need to fix that. But I feel that "inefficient but works given enough memory" is less urgent than "crashes".
[15:23] <DSpaceSlackBot1> <pbecker> @mwood that was the idea behind my PR that just removed the uncachEntity() call within the authority indexer.
[15:23] <DSpaceSlackBot1> <mwood> We also need a model to point to when someone finds another bulk operation that wastes memory.
[15:23] <DSpaceSlackBot1> <tdonohue> The problem is 3681 is a "known bug" with no definitive replication steps. I agree that there's concerns that it may be using a lot of memory, but we have no definitive description of when issues might occur.
[15:24] <DSpaceSlackBot1> <tdonohue> So, I agree with 3681 conceptually, and I want to see it move forward. But, I don't see any reason to make a big deal out of it until someone encounters it
[15:24] <DSpaceSlackBot1> <pbecker> @tdonohue I'm already conviced to delay 3681 and push out 6.2 immediately. ;)
[15:25] <DSpaceSlackBot1> <mwood> OK. Let's get 3618 out "soon" but not wait on it for 6.2.
[15:25] <DSpaceSlackBot1> <tdonohue> @pbecker : I was responding to the note about adding 3681 to documentation as a "known bug", as I don't think it really is "known" yet...we don't know how to trigger it quite yet other than "it likely will hit memory issues at some point"
[15:25] <DSpaceSlackBot1> <pbecker> ah. got it.
[15:26] <DSpaceSlackBot1> <tdonohue> But, let's just move back to 6.2 discussions... I think we just set 3681 aside for now. We do need to move it forward, but we'll do so after 6.2
[15:26] <DSpaceSlackBot1> <hpottinger> well, you could compare memory footprints using YourKit or some other profiler
[15:26] <DSpaceSlackBot1> <tdonohue> @hpottinger : no one has done that yet though, which is my point. It's not yet "known" how severe this is ;)
[15:26] <DSpaceSlackBot1> <mwood> So how many items are in the repo. on which the problem was discovered? That's a rough upper bound on conditions to reproduce it.
[15:27] <DSpaceSlackBot1> <pbecker> @hpottinger isn't that the point where you normally jump in and ask if the someone volunteer to do so?
[15:27] <DSpaceSlackBot1> <tdonohue> @mwood : Unless I'm mistaken, 3681 hasn't been "discovered" on any repo yet. It was a discovery that the code looks like it'd load everyting into memory and hit issues
[15:28] <DSpaceSlackBot1> <hpottinger> I'm pretty sure I already volunteered, and I am just saying how I plan to do it :slightly_smiling_face:
[15:29] <DSpaceSlackBot1> <tdonohue> It sounds like we all agree here...we're just talking around how to move 3681 forward. The main points here are though that we should (1) Move forward with 6.2 as-is, (2) figure out ways to test/reproduce 3681 and move its PR forward post-6.2
[15:29] <DSpaceSlackBot1> <pbecker> I remove the call to uncacheEntity(). I felt obligated to at least create a ticket about that. @sulfrian was so kind to create a PR.
[15:30] <DSpaceSlackBot1> <hpottinger> OK, so... 3681 has a PR, but it doesn't look like it's linked on the ticket...
[15:30] <DSpaceSlackBot1> Action: hpottinger starts scrolling back
[15:30] <DSpaceSlackBot1> <tdonohue> So, back to 6.2 scheduling then. @mwood is there a day that works better for you to cut the 6.2 release? (I'm around this week to help as needed too)
[15:30] <DSpaceSlackBot1> <pbecker> https://github.com/DSpace/DSpace/pull/1835
[15:30] <DSpaceSlackBot1> <tdonohue> @hpottinger : You need to be logged into JIRA. Then you'll see the "development" section on 3681 which links to the PR that @pbecker just posted.
[15:31] <DSpaceSlackBot1> <mwood> I have one 15-minute meeting on Friday. Same on Thursday plus the DSpace 7 meeting.
[15:31] <DSpaceSlackBot1> <hpottinger> DSPR#1835
[15:32] <DSpaceSlackBot1> <tdonohue> @mwood : It's your call really. You could start it post- DSpace 7 on Thursday... or first-thing Friday
[15:32] <DSpaceSlackBot1> <tdonohue> (I'd recommend starting it earlier in the day, in case you need extra time to resolve issues)
[15:32] * ksclarke (~ksclarke@pdpc/supporter/active/ksclarke) has joined #duraspace
[15:33] <DSpaceSlackBot1> <mwood> I like that: start Thursday afternoon, with Friday available as needed.
[15:33] <DSpaceSlackBot1> <tdonohue> Ok, sounds good to me then. @mwood will cut DSpace 6.2 tomorrow afternoon (with overflow into Fri if needed)! Thanks @mwood!
[15:34] <DSpaceSlackBot1> <tdonohue> And, let me know if I can help in any way. I'll be around a bit, and may be able to chip in on Release Notes, etc, as needed (although those are already started at https://wiki.duraspace.org/display/DSPACE/DSpace+Release+6.2+Status)
[15:35] <DSpaceSlackBot1> <mwood> Thanks!
[15:35] <DSpaceSlackBot1> <tdonohue> So, that wraps up 6.2 planning.
[15:36] <DSpaceSlackBot1> <tdonohue> I'll briefly mention that 5.8 was released late last week. I forgot to get the announcement sent out yesterday, but will do so today. https://github.com/DSpace/DSpace/releases/tag/dspace-5.8
[15:36] <DSpaceSlackBot1> <tdonohue> Thanks to @terrywbrady (and @mwood) for the 5.8 release!
[15:37] <DSpaceSlackBot1> <tdonohue> The rest of our agenda for today is a bit open ended (purposefully). We have several "ongoing discussion topics" (see bullet #3) that have been hanging on here for some time: https://wiki.duraspace.org/display/DSPACE/DevMtg+2017-09-06
[15:37] <DSpaceSlackBot1> <tdonohue> Are there any we'd like to talk through in the remaining 20+ minutes?
[15:38] <DSpaceSlackBot1> <tdonohue> I think the DSpace Database Connections discussion (3.a.) has been discussed a bit already, and docs have begun at https://wiki.duraspace.org/display/DSPACE/DSpace+Database+Access So, that might be one we can take off this list for now
[15:38] <DSpaceSlackBot1> <tdonohue> (unless we want to talk DSpace 7 plans for that)
[15:38] <DSpaceSlackBot1> <pbecker> I have a question regarding 3aII and 3aIII In that list.
[15:38] <DSpaceSlackBot1> <tdonohue> @pbecker : go for it
[15:39] <DSpaceSlackBot1> <pbecker> Two context object within the same thread: does that mean within the same web/cli session?
[15:39] <DSpaceSlackBot1> <pbecker> Can we be sure that to different user never get the same DB Connection?
[15:39] <DSpaceSlackBot1> <tdonohue> @pbecker: no...each *request* is a new thread, so connections won't be shared across user requests
[15:40] <DSpaceSlackBot1> <tdonohue> But connections *will* be shared within a single request (which is not the same behavior as in 5.x)
[15:40] <DSpaceSlackBot1> <mwood> There is a way to ask Hibernate for a fresh Session, but we don't provide access to it in DSpace.
[15:40] <DSpaceSlackBot1> <pbecker> hopefully no one will create a servlet container that reuses threads.
[15:40] <DSpaceSlackBot1> <pbecker> Do we have a ticket on what @mwood just pointed out?
[15:40] <DSpaceSlackBot1> <tdonohue> Hibernate Session != HTTP Session. We need to be careful about saying "session" here
[15:41] <DSpaceSlackBot1> <mwood> I believe that there is code to ensure that the current transaction is closed at the end of a request.
[15:41] <DSpaceSlackBot1> <mwood> Oops, yes, Hibernate Session.
[15:41] <DSpaceSlackBot1> <tdonohue> Hibernate Session is loosely a *database connection* from the DB pool
[15:41] <DSpaceSlackBot1> <pbecker> fine. that were my two cents.
[15:41] <DSpaceSlackBot1> <pbecker> thanks for the answers
[15:42] <DSpaceSlackBot1> <tdonohue> Ok, So, regarding these database connection issues. I think we should continue to enhance the "DSpace Database Access" docs to describe the new behavior in 6.x. But, I'm going to take this off the discussion topic list, as I'm not sure we have anything to discuss in detail here _quite yet_
[15:43] <DSpaceSlackBot1> <tdonohue> We can pull this topic back into discussion as we see fit though, obviously
[15:43] <DSpaceSlackBot1> <pbecker> @tdonohue fine with me. Do we need a ticket for DSpace 7 to check if we handle Hibernate sessions correctly and whether we need to expose the methods to handle hibernate session towards DSpace?
[15:44] <DSpaceSlackBot1> <mwood> I think that first we need a theory of how we should use Hibernate, and then figure out what code changes are needed.
[15:45] <DSpaceSlackBot1> <tdonohue> Yes, I think maybe that is the discussion topic now that I think about it... how do we want this to behave in DSpace 7 (i.e. are we OK with the DSpace 6 way, or do we see areas of improvement), and start to map out what code changes we'd need
[15:45] <DSpaceSlackBot1> <mwood> We have been running into problems with code that was converted for Hibernate mechanically but not logically.
[15:45] <DSpaceSlackBot1> <tdonohue> Currently DSpace 7 is synced with DSpace 6... in that 7 will behave the same as 6 does. We have the opportunity for larger refactors in the DSpace 7 codebase, but we don't have clear proposals on what those changes should be
[15:46] <DSpaceSlackBot1> <tdonohue> So, I'll leave this topic around but update it...we need to talk ideas specific to DSpace 7.
[15:46] <DSpaceSlackBot1> <tdonohue> But, for today, I'd like to see if we can get a few of these other "hanging" topics talked through briefly
[15:47] <DSpaceSlackBot1> <mwood> OK
[15:47] <DSpaceSlackBot1> <pbecker> +1
[15:47] <DSpaceSlackBot1> <tdonohue> So, I see this question of DSpace + Spring (3.b.). Personally, I don't see any question here...the Java API is moving towards Spring (during the 6.0 refactor and now again in the 7.0 REST API). So, I think the answer is simply "Yes, we should move DSpace more towards Spring"
[15:48] <DSpaceSlackBot1> <tdonohue> But, that then begs the question: "how else should it or could it use Spring" ;)
[15:49] <DSpaceSlackBot1> <mwood> More use of dependency injection. Right now, some code calls "new DSpace().getThis().getThat()", other code has to know which ThingServiceFactory.getInstance().getWhatsit() to call, and some lucky code is beans which can just use @Inject.
[15:49] <DSpaceSlackBot1> <pbecker> Spring security, authN/Z
[15:50] <DSpaceSlackBot1> <mwood> I think we should let Spring inspect all of our classes so that we can use @Inject more liberally. This may require some refactoring.
[15:50] <DSpaceSlackBot1> <tdonohue> @mwood : yes, I agree with all that. I guess the question in the agenda doesn't make sense though... It sounds more like we should generally be creating tickets (or proposals) of other areas of the codebase that could use a refactor to better make use of Spring
[15:51] <DSpaceSlackBot1> <mwood> You may be right.
[15:52] <DSpaceSlackBot1> <tdonohue> So, I think this question is not the correct one. We are already using Spring, and we should start to embrace that more. We all should start to look more closely at areas of the code that are "old way" or "only partially refactored" and propose ways to make them "Spring way" in either DSpace 7 or beyond
[15:52] <DSpaceSlackBot1> <pbecker> @tdonohue what is with spring security for AuthN/Z?
[15:53] <DSpaceSlackBot1> <sulfrian> @pbecker There was some discussion in github regarding auth in DSpace 7.
[15:53] <DSpaceSlackBot1> <tdonohue> @pbecker : I'd embrace that...but it'd be a major refactor of our existing AuthN/Z, so I'm not sure myself it's possible in time for DSpace 7. But, maybe for DSpace 8?
[15:54] <DSpaceSlackBot1> <sulfrian> Here: https://github.com/DSpace/Rest7Contract/issues/10
[15:54] <DSpaceSlackBot1> <mwood> We should look at widely-used things like Spring Security to see if we can drop our home-grown AuthNZ code.
[15:54] <DSpaceSlackBot1> <pbecker> I would add it to the list of thing where we can/should move into the direction of spring. Or is that limited to DSpace 7?
[15:54] <DSpaceSlackBot1> <pbecker> @sulfrian thank you!
[15:55] <DSpaceSlackBot1> <tdonohue> For DSpace 7, we are planning to use the same DSpace 6 AuthN/Z (with a Spring Security "shim" on the REST API layer), as we worry replacing our entire custom AuthN/Z is a massive endeavor (and on DSpace 7 team, we don't have enough resources to take that on right now)
[15:55] <DSpaceSlackBot1> <tdonohue> But, from what I've heard *everyone* agrees our custom AuthN/Z needs to go (sooner the better) and be replaced likely by Spring Security
[15:55] <DSpaceSlackBot1> <pbecker> @tdonohue would you prefer someone to chip into DSpace7 or into Spring Security in our java backend?
[15:56] <DSpaceSlackBot1> <mwood> What AuthNZ needs more than replacing our own code is to take decisions away from the user interfaces and move them into core objects, so that questions are decided the same way no matter how they are asked.
[15:56] <DSpaceSlackBot1> <tdonohue> @pbecker : I'd rather have more resources for DSpace 7 right now...as the sooner we get the new UI done the better (and it allows us *all* to work together instead of on separate UIs)
[15:58] <DSpaceSlackBot1> <tdonohue> Unfortunately, I'm going to have to run shortly here to a DSpace Steering Mtg
[15:58] <DSpaceSlackBot1> <tdonohue> But, I think we all agree that DSpace + Spring is the future here...we just need more ideas/proposals/tickets on where we can better use Spring in our code
[15:59] <DSpaceSlackBot1> <tdonohue> So, I need to jump to another meeting now, but it's been a good discussion.
[15:59] <DSpaceSlackBot1> <tdonohue> And I'll be around all week to help with 6.2 release, etc
[16:00] <DSpaceSlackBot1> <tdonohue> Have a good one! (Feel free to continue the discussion if you want, but I'll just be lurking)
[16:00] <DSpaceSlackBot1> <mwood> DS-2931 may be relevant to Spring discussion.
[16:00] <DSpaceSlackBot1> <pbecker> I have to run too. Good bye everyone.
[19:00] * dyelar (~dyelar@129.237.146.129) Quit (Quit: Leaving.)
[20:46] * ksclarke (~ksclarke@pdpc/supporter/active/ksclarke) Quit (Ping timeout: 248 seconds)
[21:01] * mhwood (~mhwood@mhw.ulib.iupui.edu) Quit (Remote host closed the connection)
[21:39] * tdonohue (~tdonohue@dspace/tdonohue) Quit (Read error: Connection reset by peer)
These logs were automatically created by DuraLogBot on irc.freenode.net using the Java IRC LogBot.