Timestamps are in GMT/BST.
[1:30] * peterdietz (uid52203@gateway/web/irccloud.com/x-dbrlohnjypqlalel) Quit (Quit: Connection closed for inactivity)
[6:36] -morgan.freenode.net- *** Looking up your hostname...
[6:36] -morgan.freenode.net- *** Checking Ident
[6:36] -morgan.freenode.net- *** Found your hostname
[6:36] -morgan.freenode.net- *** No Ident response
[6:36] * DuraLogBot (~PircBot@webster.duraspace.org) has joined #duraspace
[6:36] * Topic is '[Welcome to DuraSpace - This channel is logged - http://irclogs.duraspace.org/]'
[6:36] * Set by cwilper!ad579d86@gateway/web/freenode/ip.173.87.157.134 on Fri Oct 22 01:19:41 UTC 2010
[7:43] * awoods (~awoods@c-98-245-50-182.hsd1.co.comcast.net) Quit (Ping timeout: 244 seconds)
[7:44] * awoods (~awoods@c-98-245-50-182.hsd1.co.comcast.net) has joined #duraspace
[9:05] * pbecker (~pbecker@ubwstmapc098.ub.tu-berlin.de) has joined #duraspace
[13:13] * mhwood (mwood@mhw.ulib.iupui.edu) has joined #duraspace
[13:41] * tdonohue (~tdonohue@c-98-226-113-104.hsd1.il.comcast.net) has joined #duraspace
[14:08] * srobbins (~srobbins@130.126.32.172) Quit (*.net *.split)
[14:15] * srobbins (~srobbins@130.126.32.172) has joined #duraspace
[14:21] * peterdietz (uid52203@gateway/web/irccloud.com/x-qzvzmqqojwtaroxy) has joined #duraspace
[16:25] * hpottinger (~hpottinge@mu-160190.dhcp.missouri.edu) has joined #duraspace
[17:25] * terry-b (~chrome@71-35-103-116.tukw.qwest.net) has joined #duraspace
[17:41] * pbecker (~pbecker@ubwstmapc098.ub.tu-berlin.de) Quit (Quit: Leaving)
[20:00] <tdonohue> Hello all, it's time for our weekly DSpace Developers Mtg. Agenda: https://wiki.duraspace.org/display/DSPACE/DevMtg+2015-11-11
[20:01] <kompewter> [ DevMtg 2015-11-11 - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DevMtg+2015-11-11
[20:01] <tdonohue> Today will mostly be about 6.0 topics (and maybe we'll even get into a few 6.0 PR reviews)
[20:01] <tdonohue> But first, just wanted to *thank* everyone for their help / involvement in getting 5.4 (and 4.4 and 3.5) out the door this week!
[20:02] <hpottinger> likewise, thank YOU tdonohue.
[20:02] <mhwood> +1
[20:02] <tdonohue> I think 5.4 went a long way to help stabilize 5.x...and now we all get to concentrate more heavily on 6.0
[20:02] <tdonohue> no problem :)
[20:03] <tdonohue> So, we might as well move right along to 6.0 stuff today.
[20:03] <tdonohue> First, I thought this might be a good opportunity to review our timelines for 6.0. Namely, our deadline for feature PRs is TOMORROW: https://wiki.duraspace.org/display/DSPACE/DSpace+Release+6.0+Status#DSpaceRelease6.0Status-TimelineandProcessing
[20:03] <mhwood> pbecker couldn't be here today, but wanted to draw attention to several 6.0 blockers he's filed recently.
[20:03] <kompewter> [ DSpace Release 6.0 Status - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DSpace+Release+6.0+Status#DSpaceRelease6.0Status-TimelineandProcessing
[20:04] <tdonohue> mhwood: ok, we can get to those in a bit
[20:05] <tdonohue> Regarding the timeline, I guess the question is...is there anything *outstanding* that won't be ready for review by tomorrow?
[20:05] <tdonohue> I'll note that my Commons Config work *should* have a PR that is in "ready for review" state by end of day tomorrow. It's 99% there, I think
[20:06] * tdonohue has a doorbell ringing (just a sec)
[20:06] <tdonohue> back
[20:07] <tdonohue> So, is there anything else outstanding that I'm *not* aware of? Or do we feel we should stick with the PR deadline of tomorrow?
[20:08] <tdonohue> Ok, it sounds like we likely stick to our deadline. If any PRs need more time, we can look to extend things for individuals if absolutely needed
[20:09] <tdonohue> As mhwood mentioned it, we do also have some 6.0 Blockers coming in that need help/investigation
[20:10] * tdonohue is looking for a link to them
[20:10] <tdonohue> Here they are: https://jira.duraspace.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+DS+AND+fixVersion+%3D+6.0+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker+ORDER+BY+key+DESC&mode=hide
[20:10] <kompewter> [ Issue Navigator - DuraSpace JIRA ] - https://jira.duraspace.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+DS+AND+fixVersion+%3D+6.0+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker+ORDER+BY+key+DESC&mode=hide
[20:11] <mhwood> Pascal notes that JSPUI feels buggy.
[20:12] <tdonohue> mhwood: any other particular notes, beyond the blockers?
[20:12] <mhwood> That's all I had.
[20:12] <tdonohue> ok. Just checking
[20:12] <mhwood> He mentioned it Monday in IRC and I just scribbled a note to bring it up.
[20:13] <tdonohue> I'm not sure if it's worth our time to dig into these blockers *now*. But, I do encourage folks to try and help chip in / test these same things over in XMLUI. It's possible some of these are issues at the API level, and not necessarily just JSPUI
[20:13] <hpottinger> I wonder if the draft DCAT checklist would be usable for thoroughly testing our two UIs?
[20:13] <tdonohue> Plus, I'd encourage others to log blockers if you are hitting on major issues/bus
[20:13] <tdonohue> *bugs
[20:13] <tdonohue> hpottinger: have a link handy?
[20:14] <hpottinger> I bet terry-b can beat me to it
[20:15] <tdonohue> In general though, I imagine it could be useful. At the very least, it might be worth testing some of these JSPUI blocks in XMLUI as well...it'll help us narrow down the problem
[20:15] <hpottinger> here's the workgroup page: https://wiki.duraspace.org/display/cmtygp/DSpace+6+Testathon+Testplan+Working+group
[20:15] <kompewter> [ DSpace 6 Testathon Testplan Working group - Community Groups - DuraSpace Wiki ] - https://wiki.duraspace.org/display/cmtygp/DSpace+6+Testathon+Testplan+Working+group
[20:15] <tdonohue> s/blocks/blockers
[20:15] <kompewter> tdonohue meant to say: In general though, I imagine it could be useful. At the very least, it might be worth testing some of these JSPUI blockers in XMLUI as well...it'll help us narrow down the problem
[20:15] <hpottinger> https://wiki.duraspace.org/display/DSPACE/DSpace+Release+6.0+Testathon+Page
[20:15] <kompewter> [ DSpace Release 6.0 Testathon Page - DSpace - DuraSpace Wiki ] - https://wiki.duraspace.org/display/DSPACE/DSpace+Release+6.0+Testathon+Page
[20:16] <tdonohue> cool!
[20:17] <tdonohue> In any case, I don't know that I want to spend too much more time on this specific topic today. There's a lot to talk about, and considering we aren't even under a "feature freeze" we'll have plenty of testing coming up
[20:17] <hpottinger> there is more stuff somewhere, I know Edinburgh has a detailed list they use
[20:18] <hpottinger> Oh, I see, "Different people's own test plans"
[20:18] <tdonohue> So, I'm going to move along for now to another general topic for 6.0. The question of whether we think about deprecating Elastic Search in 6.0 for later removal (likely in 7.0)
[20:18] <mhwood> helix84 also couldn't be here, and wanted to note that ES sites like the charts that come with that stat. implementation. He suggests to port the charting to Solr if/when ES is removed. I think it would be good to go ahead with that regardless of ES' fate.
[20:19] <hpottinger> mhwood++
[20:19] <hpottinger> helix84++
[20:19] <tdonohue> mhwood++ as well. I agree, it'd really just need a volunteer to make that happen. I'm not sure of the effort though
[20:20] <tdonohue> So, I guess the first question is: Do we all agree that ES should be deprecated? If so, then we need some ticket(s) created, and one of those should be able porting ES charts to Solr Stats
[20:22] <mhwood> It might be hard to support if Longsight is dropping it.
[20:22] <tdonohue> So, let's do a (non-binding) vote of those here. Those in favor of deprecating ES in 6.0?
[20:22] <hpottinger> it's functionally deprecated now, isn't it?
[20:22] <mhwood> +1
[20:22] <hpottinger> +1 deprecation
[20:22] <tdonohue> +1 deprecation
[20:23] <terry-b> +1
[20:23] <tdonohue> Ok. sounds like we need to formally deprecate it in 6.0 & work to replace it in 7.0. I'll create a ticket for both steps here, and schedule them. If anyone disagrees, you can add notes to the ticket itself
[20:24] <mhwood> Sounds good.
[20:24] <mhwood> Thanks!
[20:24] <hpottinger> do we need to do a e-mail list vote on the deprecation?
[20:24] <mhwood> Probably, after the tickets go up.
[20:25] <tdonohue> I'll let folks know we are proposing deprecation. If there's argument, we'll talk about it. But, unless we find another group to support it, I don't see how we can *not* deprecate it
[20:25] <mhwood> That way there is a place to record discussion.
[20:26] <tdonohue> Ok. So, I'll take care of those tickets & email to dspace-devel later today or tomorrow
[20:27] <tdonohue> As for other 6.0 topics. I feel it necessary to chat a bit about Commons Config. The changes here are *massive* (maybe not as massive as Services API, but still very large). They also are *not* going to be backwards compatible with old ways of doing configs.
[20:27] <tdonohue> DSPR#1104 is the work...it's 99% done, but I need to do docs still
[20:27] <kompewter> [ https://github.com/DSpace/DSpace/pull/1104 ] - DS-2654: Reloadable Configurations via Apache Commons Configuration by tdonohue
[20:28] <tdonohue> There's a few key things to be aware of:
[20:28] <tdonohue> (1) Build.properties is completely gone. It's been replaced by a new "local.cfg" file which can be used to override *any* config setting from *any* cfg file (even under /modules)
[20:28] <mhwood> [applause]
[20:29] <tdonohue> Here's a sample https://github.com/tdonohue/DSpace/blob/DS-2654-common-config/local.cfg.EXAMPLE
[20:29] <terry-b> That is fantastic and will greatly simplify upgrades!
[20:29] <kompewter> [ DSpace/local.cfg.EXAMPLE at DS-2654-common-config · tdonohue/DSpace · GitHub ] - https://github.com/tdonohue/DSpace/blob/DS-2654-common-config/local.cfg.EXAMPLE
[20:29] <kompewter> [ https://jira.duraspace.org/browse/DS-2654 ] - [DS-2654] Reloadable Configurations via Apache Commons Configuration - DuraSpace JIRA
[20:30] <tdonohue> There's a new "local.cfg.EXAMPLE" which can be used to "start" your dspace.cfg. It looks kinda like build.properties, but is something *completely* different (as you can add any config you want to it)
[20:30] <tdonohue> s/dspace.cfg/local.cfg
[20:30] <kompewter> tdonohue meant to say: There's a new "local.cfg.EXAMPLE" which can be used to "start" your local.cfg. It looks kinda like build.properties, but is something *completely* different (as you can add any config you want to it)
[20:31] <tdonohue> The DSpace build/compilation process DOES NOT USE "local.cfg". It's NOT like how build.properties was built heavily into the build/compilation
[20:32] <tdonohue> local.cfg *is* used by Ant though...*only* to get the "dspace.dir". Technically, if we required specifying that directory to "ant update", then local.cfg wouldn't even need to be used there
[20:32] <mhwood> [applause]
[20:33] <tdonohue> All the crazy weird "filtering" of values (of things like ${dspace.dir}) is also COMPLETELY GONE. So, your final "dspace.cfg" and "local.cfg" *can* include ${variables} and everything will work perfectly. All the filtering is removed
[20:33] <hpottinger> this is pretty cool, exciting work, tdonohue
[20:33] <mhwood> [WILD APPLAUSE]
[20:34] <tdonohue> I literally ripped out all the filtering from both Maven & Ant. It's all gone
[20:34] <tdonohue> To get this all to work, there are a few "gotchas" though.... the biggest one is: Some configuration settings HAD TO BE RENAMED
[20:35] <tdonohue> Unfortunately, we can no longer have two settings named "enabled=true" which are in separate *.cfg files...they all had to be prepended with a prefix in order to make config keys unique
[20:36] <tdonohue> So, this does mean the upgrade of your configuration settings will be slightly painful (but then *much* easier in the future if you use the local.cfg)
[20:37] <mhwood> I can live with that. Naming of config. settings has been rather chaotic. Maybe we need to create some written guidelines.
[20:38] <tdonohue> I'll also note that even though this is called "Reloadable" configuration (we may need to rename that for 6.0)... the actual "reloadable-ness" of individual configs is currently NOT guaranteed. Technically, since we are using Commons Configs, individual settings *are* reloadable. But, DSpace has a bad habit of storing config settings in "static" fields which *never* actually re-loading the config value in code
[20:38] <tdonohue> I had to draw the line at tracking down all those crazy "static" fields. It's too hard, and I think there are a lot of them
[20:39] <cwilper> tdonohue: looks interesting, good to move to third party utils for doing this type of stuff. one thing i think people will be curious about is what the new pattern is for environment-specific settings. (replacement for -Denv=prod/test/etc).
[20:39] <tdonohue> But, even without the full "reloadable-ness"... the addition of the local.cfg is a MAJOR improvement (IMHO)
[20:39] <terry-b> Could a tool be written to facilitate the migration of settings - scan an active config and dump all settings that re not default.
[20:39] <mhwood> Any code that uses ConfigurationManager likely was written that way. Code that uses ConfigurationService is more likely to do so dynamically since CS was written with reloadability in mind.
[20:40] <tdonohue> cwilper: thanks for the prompt. In our Apache Commons Config settings, I've set it up so that you can override ANY config from environment variables or System properties. So, you can do -Ddspace.dir... and it'll override what is in your local.cfg/dspace.cfg
[20:41] <tdonohue> The ordering is the System Properties overrides everything, then environment variables, then local.cfg, finally dspace.cfg (and other /modules/*.cfg)
[20:42] <tdonohue> But, that ordering can be completely customized in a new "config-definition.xml" file: https://github.com/tdonohue/DSpace/blob/DS-2654-common-config/dspace/config/config-definition.xml
[20:42] <kompewter> [ DSpace/config-definition.xml at DS-2654-common-config · tdonohue/DSpace · GitHub ] - https://github.com/tdonohue/DSpace/blob/DS-2654-common-config/dspace/config/config-definition.xml
[20:42] <kompewter> [ https://jira.duraspace.org/browse/DS-2654 ] - [DS-2654] Reloadable Configurations via Apache Commons Configuration - DuraSpace JIRA
[20:42] <mhwood> modules/*.cfg are all in the same namespace with dspace.cfg now, yes?
[20:42] <tdonohue> mhwood: correct. now the "dspace.cfg" essentially "loads" the modules/*.cfg files within itself
[20:43] <mhwood> So we can have yet more elaborate schemes if we choose. I foresee some simplification of the test environment setup....
[20:43] <tdonohue> As I mentioned earlier in the meeting, my code is nearly all at a state where you can pull it down and try it out. There's a few things outstanding:
[20:44] <tdonohue> (1) I need to fix web.xml files to no longer require filtering (I know the fix, just haven't done it yet).. Currently they cannot figure out what ${dspace.dir} means
[20:44] <tdonohue> (2) I need to provide more documentation on how to get started and what all has changed
[20:44] <tdonohue> everything else *works*
[20:44] <mhwood> terry-b: I think it shouldn't be hard to read in two .properties and output the diff.
[20:45] <hpottinger> tdonohue: how can we help?
[20:46] <tdonohue> back to cwilpers note here... I should be clear though that the "-Denv" setting is *gone* from the build/compilation process. That was an artifact of build.properties, and as I mentioned "local.cfg" isn't used in the build/compilation. But, you *can* now specific -D[anysetting]=[newvalue] during *runtime* (in Tomcat, etc) and it'll override the value in you config files
[20:47] <tdonohue> hpottinger: i think the biggest help will be in getting testers/feedback. I wanted to start today in getting any feedback/questions and making it clear what *is* and *is not* included in this change
[20:47] <mhwood> I will not miss the ever-growing list of things that *had to* be customized in build.properties.
[20:48] <tdonohue> I can also let everyone know once the PR is ready for testing (should be tomorrow)... the docs will likely be very much a Work in progress for some time. But, I can quickly write up the basics of getting started
[20:49] <tdonohue> I think that's all I wanted to say...and sorry to have taken up so much time...but I think this PR is quite important to get feedback on
[20:49] <cwilper> tdonohue: happy to kick the tires a bit. we typically have 3 to 5 different environments (local/ci/staging/prod) per dspace instance we manage.
[20:50] <mhwood> It is quite a change, and one that will be more visible to installations than the *other* big change in 6.0.
[20:50] <hpottinger> I'll need to experiement with different ways to reproduce the functionality of env.properties, I think ENV Variables would probably work (and not expose sensitive info in a process listing) but I'll need to tinker to be sure
[20:50] <tdonohue> cwilper: I'd encourage you to definitely kick the tires. You already can take an early look at the PR...but the web.xml's will be ever so slightly broken till I finish that fix
[20:51] <mhwood> Here we just stomp all over the web.xml value of dspace.dir with <Parameter>s in the <Context>.
[20:52] <tdonohue> hpottinger: I did worry initially about ditching "-Denv=[]" settings...but, in the end, I think Commons config offers *better* (more standard) ways to do similar things. You could now set configs in your CATALINA_OPTS, for example
[20:52] <mhwood> I only recently became aware that there was even a value in there.
[20:53] <hpottinger> right, I just need to figure out how to do it the "better" way :-)
[20:53] <tdonohue> Any last questions on the Commons Config stuff that I can clarify? (If not, we can move along to other things)
[20:54] <cwilper> more standard = good. i think there may be a way to still do something -Denv-like in the new way, but i want to get a better sense of what's at our disposal in the branch as-is before suggesting changes.
[20:54] <hpottinger> I suspect I may need to approach this like puppet does hiera, and specify a new cfg file
[20:54] <tdonohue> hpottinger: yes, there may be a small learning curve with that...especially for folks who relied heavily on the "-Denv" build settings. But, I think we'll come up with much better ways by sharing notes
[20:55] <tdonohue> It *is* possible to add more config files by specifying them in your "config-definition.xml" (I linked to it above). By default, that just looks for a "local.cfg" and a "dspace.cfg" though
[20:55] <mhwood> I eventually threw in the towel and wrote a script to write systematically customized whatever.properties, but that's easily converted to writing .cfg files and now I don't have to be always looking over my shoulder to see what new settings have been added (that will break installation until I fix my script).
[20:56] <tdonohue> There's essentially a *ton* of flexibility we are gaining in this route...and honestly, most of what I've done is rip out a bunch of code which was hacking around the "build.properties" concept to make it work "kinda like" Commons Config (in overriding config values)
[20:56] <hpottinger> what I meant when I said "like puppet does hiera": https://github.com/DSpace/vagrant-dspace/blob/a9fb0875997f9128537b2564067c4821531ec815/hiera.yaml
[20:56] <kompewter> [ vagrant-dspace/hiera.yaml at a9fb0875997f9128537b2564067c4821531ec815 · DSpace/vagrant-dspace · GitHub ] - https://github.com/DSpace/vagrant-dspace/blob/a9fb0875997f9128537b2564067c4821531ec815/hiera.yaml
[20:58] <tdonohue> hpottinger: if you look closely at the config-definition.xml file...it's *kinda* like a hiera.yml. But, it's an XML file so it's a ton more wordy https://github.com/tdonohue/DSpace/blob/DS-2654-common-config/dspace/config/config-definition.xml
[20:58] <kompewter> [ DSpace/config-definition.xml at DS-2654-common-config · tdonohue/DSpace · GitHub ] - https://github.com/tdonohue/DSpace/blob/DS-2654-common-config/dspace/config/config-definition.xml
[20:58] <kompewter> [ https://jira.duraspace.org/browse/DS-2654 ] - [DS-2654] Reloadable Configurations via Apache Commons Configuration - DuraSpace JIRA
[20:58] <tdonohue> hpottinger: but the end result is that config-definition.xml *tells* Apache Commons Configuration where to find the configs. You can even tell it to load them from a *database* in that config file
[20:59] <hpottinger> looking at it now, I think we would just need a middle layer between local and dspace cfg
[20:59] <hpottinger> loca, usual and customary, and dspace cfgs
[21:00] <mhwood> If you strain out the comments, there's really very little there. Not wordy at all.
[21:00] <tdonohue> hpottinger: in case you want to dig deeper, this "config-definition.xml" is a standard config file for Apache Commons Config. The full docs of *everything* you could do within it are here: http://commons.apache.org/proper/commons-configuration/userguide_v1.10/howto_configurationbuilder.html
[21:00] <kompewter> [ Commons Configuration – Configuration Builder Howto ] - http://commons.apache.org/proper/commons-configuration/userguide_v1.10/howto_configurationbuilder.html
[21:01] <mhwood> (I used to write COBOL, though, and still do on occasion, so my idea of "wordy" may be unusual today.)
[21:01] <tdonohue> As I said, Apache Commons Config even allows you to load configs from a database backend, XML config files, etc.... we're just using the Property-style config settings, as that is what we use
[21:02] <cwilper> gtg for now, but definitely want to compare notes with ya'll on strategies for doing env-specific stuff. another thought is just to use a symlink in each clone the points to the appropriate actual local.cfg for that environment. all in all, i'm glad to see code getting ripped out.
[21:02] <mhwood> Mmmm...JNDI....
[21:02] <tdonohue> In any case...we've talked our way through a whole hour :)
[21:03] <tdonohue> Next week, we'll need to concentrate on reviewing 6.0 PRs. Obviously though, please keep reviewing PRs of high-importance offline & merging them once they get a few +1's
[21:03] <tdonohue> Any final thoughts/questions for today?
[21:04] <hpottinger> save me some backtracking and link your PR again?
[21:05] <hpottinger> too late, I found it: https://github.com/DSpace/DSpace/pull/1104
[21:05] <kompewter> [ DS-2654: Reloadable Configurations via Apache Commons Configuration by tdonohue · Pull Request #1104 · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/pull/1104
[21:05] <tdonohue> OK, we'll close up the meeting for today. Thanks all!
[21:18] <mhwood> Anyone looking for something simple to review may wish to have a look at DSPR#1019 :-)
[21:18] <kompewter> [ https://github.com/DSpace/DSpace/pull/1019 ] - [DS-2463] iplists.com-non_engines.txt is outdated and slow by mwoodiupui
[21:29] <hpottinger> mhwood: I think 1019 looks fine, and just porting the user agent strings from the old file works for me
[21:29] <hpottinger> (the approach, I haven't tested)
[21:29] <mhwood> Thanks, glad to hear it.
[21:29] <hpottinger> (but I see no reason why it wouldn't work)
[21:41] <hpottinger> 1019 has my +1
[21:41] <mhwood> Thank you.
[22:08] * mhwood (mwood@mhw.ulib.iupui.edu) has left #duraspace
[22:46] * hpottinger (~hpottinge@mu-160190.dhcp.missouri.edu) Quit (Quit: Leaving, later taterz!)
[23:15] * tdonohue (~tdonohue@c-98-226-113-104.hsd1.il.comcast.net) has left #duraspace
These logs were automatically created by DuraLogBot on irc.freenode.net using the Java IRC LogBot.