#duraspace IRC Log


IRC Log for 2012-03-28

Timestamps are in GMT/BST.

[6:34] -niven.freenode.net- *** Looking up your hostname...
[6:34] -niven.freenode.net- *** Checking Ident
[6:34] -niven.freenode.net- *** Found your hostname
[6:34] -niven.freenode.net- *** No Ident response
[6:34] * DuraLogBot (~PircBot@atlas.duraspace.org) has joined #duraspace
[6:34] * Topic is '[Welcome to DuraSpace - This channel is logged - http://irclogs.duraspace.org/]'
[6:34] * Set by cwilper!ad579d86@gateway/web/freenode/ip. on Fri Oct 22 01:19:41 UTC 2010
[12:03] * mhwood (mwood@mhw.ulib.iupui.edu) has joined #duraspace
[16:02] * eddies (~eddies@ has joined #duraspace
[16:02] * eddies (~eddies@ Quit (Changing host)
[16:02] * eddies (~eddies@unaffiliated/eddies) has joined #duraspace
[17:28] * eddies (~eddies@unaffiliated/eddies) Quit (Quit: Leaving.)
[18:56] * hpottinger (~hpottinge@mu-162198.dhcp.missouri.edu) has joined #duraspace
[18:58] <hpottinger> PeterDietz: it struck me last night, if we're all working on our own forks of the DSpace GitHub project, are we giving up on Bamboo?
[18:58] <mhwood> We were using Bamboo?
[18:59] <hpottinger> I remember Bamboo complaining about broken builds in the past...
[18:59] <hpottinger> I might have dreamt it, though :-)
[18:59] <mhwood> GitHub has a service hook for Bamboo.
[19:00] <mhwood> And another for Jira
[19:03] * mdiggory (~mdiggory@rrcs-74-87-47-114.west.biz.rr.com) has joined #duraspace
[19:06] <hpottinger> so... we can point the DuraSpace Bamboo instance at the main DSpace GitHub project, which will catch if any pull requests break the main project
[19:06] <mhwood> It looks like we can wire DSpace/DSpace to Duraspace's Bamboo (and Jira). Your fork can use whatever you have, I guess.
[19:07] <mhwood> Yeah, when they are pulled.
[19:08] <hpottinger> right, and any committer can approve a pull request, so that's pretty much what we had before (Bamboo complaining if a commit to SVN borked the build)
[19:09] <mhwood> Yes, except that we have an opportunity for public review in personal forks before pulling to DSpace/DSpace.
[19:12] <mhwood> (Must remember to try out the Jenkins hook for my repo.s.)
[19:22] <mdiggory> Yes, I'm sure there are some Fedora configs already there we can use for a template.
[19:23] <hpottinger> making notes to look into continuous integration server setup, if anyone has a writeup, or sets this up before I do, I'd love to see notes
[19:39] <hpottinger> mhwood: where are you looking for info on the GitHub hooks (asks a very very lazy Hardy)
[19:40] <mhwood> Enter a repository. Poke the Admin button. Pick the Service Hooks tab on the left side.
[19:59] <mhwood> Successful Jenkins build of one of my repo.s after reconfiguring to fetch from GitHub. I had to add user.name and user.email to Jenkins' Git configuration -- it fails without these.
[20:02] * PeterDietzHome (~Angie@d60-65-161-223.col.wideopenwest.com) has joined #duraspace
[20:02] * PeterDietzHome (~Angie@d60-65-161-223.col.wideopenwest.com) Quit (Client Quit)
[20:03] * PeterDietzHome (413cdfa1@gateway/web/freenode/ip. has joined #duraspace
[20:04] <PeterDietzHome> hi all, just joined
[20:05] <PeterDietzHome> I haven't yet sent out an announcement, but I've finished the main DSpace to github step
[20:05] <PeterDietzHome> i.e. DSpace/DSpace got moved to DSpace/dspace-svn-deprecated or something. And the new one was renamed from DSpace/DSpaceCorrect to just DSpace/DSpace
[20:07] <PeterDietzHome> ok, so with Tim out. Any agenda ideas?
[20:07] <PeterDietzHome> perhaps someone can send in the first pull request, and fix up the readme to the git repo
[20:07] <PeterDietzHome> I am not that gifted in document markup.
[20:07] <PeterDietzHome> where's jtrimble/
[20:07] <PeterDietzHome> ?
[20:09] <mhwood> Ah, you mean the SCM links are wrong now.
[20:09] * ryscher (ae61f77a@gateway/web/freenode/ip. has joined #duraspace
[20:09] <mhwood> I can take a shot at correcting that.
[20:10] <PeterDietzHome> There's a formatting, markdown, you can use to make it show up pretty, with h1's, etc
[20:11] <mhwood> Ah, good, we get to learn Yet Another Markup Language.
[20:11] <mhwood> We need an anthropologist to work out why we keep making new markup languages and then hate them so.
[20:11] <PeterDietzHome> for those who made it to the DSUG in Missouri, any exciting take-aways for those who couldn't make it?
[20:12] <PeterDietzHome> http://xkcd.com/927/
[20:12] <kompewter> [ xkcd: Standards ] - http://xkcd.com/927/
[20:13] <PeterDietzHome> so, I would say I need to make a formal announcement to email lists, I had assumed Tim would have poked in to help craft them, but I will do this after this meeting
[20:14] <PeterDietzHome> ...but, its safe to say we are now officially using git as our official SCM
[20:14] <PeterDietzHome> ...so let the games begin
[20:14] <hpottinger> may the odds be ever in your favor
[20:15] * mhwood starts counting SVN checkouts to be converted. The price of progress, I suppose.
[20:15] <PeterDietzHome> _ever_ in your favor.. ( I haven't seen the movie yet, angie went last night, I'm only half way through the book, so no spoilers)
[20:17] <hpottinger> there's a character in the movie who really isn't a character in the first book
[20:17] <PeterDietzHome> I think we need to cover some groundwork for what _ought_ to make it into DSpace 3.0
[20:18] <PeterDietzHome> i.e. if our repository system isn't advancing, that we might be less tempted to use it, compared to alternatives.
[20:19] <PeterDietzHome> for osu, we're started to run into a question, where should we put the masters (images/tiffs) so that we can make use of them, and the curators want to keep end-users from accessing the full quality tiffs, just a reduced rendition
[20:20] <hpottinger> we put master images in a "dark" instance of DSpace
[20:21] * ryscher_ (ae61f77a@gateway/web/freenode/ip. has joined #duraspace
[20:22] * PeterDietzhome_ (413cdfa1@gateway/web/freenode/ip. has joined #duraspace
[20:22] <PeterDietzhome_> masters on the dark instance, how are the two related then? i.e. can the admin/curator click from one item on production, to the dark item behind the firewall
[20:23] * PeterDietzHome (413cdfa1@gateway/web/freenode/ip. Quit (Ping timeout: 245 seconds)
[20:23] * ryscher (ae61f77a@gateway/web/freenode/ip. Quit (Ping timeout: 245 seconds)
[20:23] <PeterDietzhome_> so.. without going too far down this rabbit hole, there might some questions that we want to have a good answer for. does DSpace solve the use case that you want it to solve
[20:25] <mhwood> The right access-control structure would let you keep public and private in the same repo. if you wanted to. Dunno how close DSpace is to that right now.
[20:26] <mhwood> Something in the back of my mind, kind of related: a way to direct some bitstreams to assetstore N and others (say, huge uncompressed TIFFs) to assetstore M, based on policies.
[20:28] <hpottinger> PeterDietzhome: masters are related via metadata, yes, though what mhwood is talking about sounds cool
[20:30] <PeterDietzhome_> I can't find the code just yet, but we've built something for tacking a citation page to the end of a document, moving the original pdf to the PRESERVATION bundle, and strip all access rights to that file/bundle, then putting the newly created citation pdf into a DISPLAY bundle
[20:30] <hpottinger> since you ask, PeterDietzhome, DSpace rocks or IR use case. It does not yet do the same for our digital library use case.
[20:30] <hpottinger> *our IR use case, that is
[20:32] <PeterDietzhome_> would digital library mean providing access to digital materials in a nice and presentable way?
[20:32] <hpottinger> correct
[20:32] <PeterDietzhome_> correct, it does not currently rock at that
[20:32] <PeterDietzhome_> ..but I think thats where something like omeka shines a bit?
[20:33] <hpottinger> Omeka, Islandora, erm.... ContentDM if you have the cash... :-)
[20:36] <PeterDietzhome_> So for 3.0, one goal would be to either tackle some aspect of the digital library (presentation), or improving the handoff to another tool to do that
[20:36] <hpottinger> backing up a bit, one exciting takeaway for us from DSUG is that there is a murmur of unhappiness with the speed of Handle.net links, but in the intervening week, we've found a solution to that
[20:38] <hpottinger> for presentation, I think your (OSU's) Snazzy theme provides a nice framework to build upon
[20:41] <PeterDietzhome_> We need to reorganize our themes.. We're running into the situation where we're making hybrid theme
[20:41] <hpottinger> I should apologize right now, I'm pretty distracted today, boss is working right next to me
[20:41] <PeterDietzhome_> a little bit of the metadata-choosing theme, a little bit of image-gallery theme
[20:42] <mhwood> Woops, missed a comment. Yes, I think DSpace as a component in a local ecosystem is an important thing to keep in mind. How do we play well with others? are we encouraging others to play well with us?
[20:42] <PeterDietzhome_> I took a half-day at work, went on a house tour, and am now typing away from home today :)
[20:45] <mhwood> It's gone quiet, so...I wanted to see if anyone had thoughts about the proposal on dspace-devel to use a crowdsourced translation service for localization?
[20:46] <PeterDietzhome_> with git we can now accept user contributions better
[20:46] <PeterDietzhome_> i don't know how unpleasant translating a few message keys out of context might be
[20:47] <PeterDietzhome_> and, without a tool to validate that all keys have been modified
[20:48] <mhwood> True, contribution is easier. But I think we might benefit from a bit more organized approach to maintaining the localizations. Yeah, such as tools to track convergence between different languages.
[20:49] <PeterDietzhome_> its worth a look, I think the pricing for open source projects was free, and if its already in use for several other big projects, I imagine they have a good system for the translators to use
[20:51] <mhwood> I don't see anyone here from Duraspace today? I think they'd have to establish the relationship, if we go that route.
[20:52] <mhwood> There seems to be at least one other such service out there -- I ran across it while exploring GitHub's service hooks (to find out what all these strange names are).
[20:52] <PeterDietzhome_> there's still some details about how/where our messages.xml/properties live, things mdiggory can help out with as we clean up and consolidate that effort
[20:53] <mdiggory> whatsup....
[20:53] <PeterDietzhome_> languages
[20:53] <PeterDietzhome_> do we want to merge xmlui-lang and api-lang back into the core?
[20:53] <mhwood> Facilitating translation, maintaining localizations, tools to track convergence....
[20:53] <mdiggory> I wasn't paying very good attentiion... how did the github migration go, you've solved all our problems correct?
[20:54] <PeterDietzhome_> ...and then, where do things like modules plug their stuff
[20:54] <mhwood> Modules have their own catalogs.
[20:54] <PeterDietzhome_> yep, I have to send the announement out, but the migration of main DSpace/DSpace went well
[20:55] <mdiggory> some do... but if we consolidate the modules, so to have we consolidated the messages
[20:55] <PeterDietzhome_> unfork the old DSpace/DSpace-svn-deprecated, cherry-pick your commits from the old repo to the new repo
[20:56] <mdiggory> so in terms of the maven project consolidation... did we feel everyone was onboard with this?
[20:56] <PeterDietzhome_> dspace-xmlui/dspace-xmlui-api to just dspace-xmlui/src/main/java/
[20:56] <mdiggory> https://github.com/mdiggory/DSpace/tree/maven-project-consolidation
[20:56] <kompewter> [ mdiggory/DSpace at maven-project-consolidation · GitHub ] - https://github.com/mdiggory/DSpace/tree/maven-project-consolidation
[20:57] <PeterDietzhome_> I'm on board with that
[20:57] <mdiggory> there are some minor concerns with project naming that tim pointed out
[20:57] <mdiggory> https://github.com/mdiggory/DSpace/tree/maven-project-consolidation/dspace-common-api
[20:57] <kompewter> [ DSpace/dspace-common-api at maven-project-consolidation · mdiggory/DSpace · GitHub ] - https://github.com/mdiggory/DSpace/tree/maven-project-consolidation/dspace-common-api
[20:58] <mdiggory> these are all the discovery, stats, swordclient application libs that support those modules all consolidated into one project
[20:58] <mdiggory> also note we left one project out
[20:58] <mdiggory> https://github.com/mdiggory/DSpace/tree/maven-project-consolidation/dspace-discovery-solr
[20:58] <kompewter> [ DSpace/dspace-discovery-solr at maven-project-consolidation · mdiggory/DSpace · GitHub ] - https://github.com/mdiggory/DSpace/tree/maven-project-consolidation/dspace-discovery-solr
[20:59] <mdiggory> this is the Solr Backend for Discovery... it too could go into common... but I left it out initially to show how the elastic-search would be packaged separately.
[20:59] <mdiggory> note, Solr is still needed for stats, so its not left out of the build...
[21:00] <mhwood> This consolidation was: for FOO in modules/*; do consolidate ${FOO}-api, ${FOO}-webapp etc into ${FOO}; done ?
[21:00] <mdiggory> all this comes back to the point that while I understand and also promote wanting to be pluggable, I do want us to simplify the build with basic default implementations included.
[21:00] <mdiggory> pretty much, there were some caveats...
[21:01] <mdiggory> and I will redo the steps as a branch in DSpace/DSpace before we place it into the master.
[21:01] <PeterDietzhome_> We don't use discovery, but how is performance of the solr search core? since its likely to be much smaller than stats, it should be faster
[21:02] <mdiggory> we are very happy with it.
[21:02] <mdiggory> and the advantages over prexisting browse are tremendous
[21:04] <mdiggory> yes, much faster than stats, but stats is pretty fast for what it is doing as well.
[21:06] <PeterDietzhome_> so part of re-org, we can set discovery as defaulted to on. much like solr stats is defaulted to on
[21:06] <mdiggory> yes we can do that.
[21:06] <PeterDietzhome_> we've completed some tools that help to migrate all your data from solr to elastic-search.. We went off the deep-end a bit
[21:06] <mdiggory> but I want to keep the two tasks separate
[21:07] <mdiggory> so, tell me about elasticsearch... how much did you implement of Discovery for it? Kevin was also at work on such a thing...
[21:07] <PeterDietzhome_> we had a race to see who could do it faster/better, and we've got a way to take Kevin's bitstream-reindex solr->csv dump, run that through a PHP script to import into elastic-search, and another implementation, written in Clojure
[21:08] <PeterDietzhome_> i got nothing for discovery yet, just a client that can be shared that talks to ES
[21:08] <PeterDietzhome_> for stats, its more or less fully capable
[21:08] <PeterDietzhome_> UsageEventListener
[21:09] <PeterDietzhome_> then a stats display page that I'm still actively developing each day
[21:09] <PeterDietzhome_> https://github.com/osulibraries/dspace-stats-elasticsearch/tree/dev/src/main/java/edu/osu/library/dspace/statistics
[21:09] <kompewter> [ dspace-stats-elasticsearch/src/main/java/edu/osu/library/dspace/statistics at dev · osulibraries/dspace-stats-elasticsearch · GitHub ] - https://github.com/osulibraries/dspace-stats-elasticsearch/tree/dev/src/main/java/edu/osu/library/dspace/statistics
[21:09] <mdiggory> where are you installing elasticsearch? someplace like ${dspace.dir}/elasticsearch
[21:10] <PeterDietzhome_> err, well, a bit out of the box here... during development its run from /Users/peterdietz/Downloads/elasticsearch-18.1/bin/elasticsearch
[21:11] <mdiggory> ah...k, not so out-of-the-box for dspace yet...
[21:11] <PeterDietzhome_> So long as you've got ES listening on localhost:9200 your golden though
[21:11] <PeterDietzhome_> but dspace doesn't install postgres
[21:12] <mhwood> Yes, we should be moving things *out* of DSpace if they don't have to live there.
[21:12] <PeterDietzhome_> I don't have as much experience as you guys do at not upsetting people with install experience
[21:14] <mhwood> Argh, must go. 'bye, all.
[21:15] * mhwood (mwood@mhw.ulib.iupui.edu) has left #duraspace
[21:15] <PeterDietzhome_> hmm, so wrap up. GITHUB migration of DSpace/DSpace has completed, look for an announcment from me some time today
[21:16] <mdiggory> most folks just want it to run, they are not concerned with where things run from... effective replication of builds generally means keeping most everything under one directory in SCM... so we usually keep the [dspace-src]/dspace/ directory under version control with solr config included.
[21:17] <PeterDietzhome_> I've got hard-coded configs for now...
[21:17] <PeterDietzhome_> public static final String clusterName = "dspacestatslogging"; public static final String indexName = "dspaceelastic"; public static final String indexType = "stats"; public static final String address = ""; public static final int port = 9300;
[21:18] <mdiggory> ok, you know I really want to port the UsageEvent configuration into dspace/config/spring rather than the way its configured right now under each webapp...
[21:19] <mdiggory> that'll ease your configuration....
[21:20] <mdiggory> this file would be emptied out
[21:20] <mdiggory> https://github.com/DSpace/DSpace/blob/master/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/WEB-INF/spring/applicationContext.xml
[21:20] <kompewter> [ DSpace/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/WEB-INF/spring/applicationContext.xml at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/blob/master/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/WEB-INF/spring/applicationContext.xml
[21:20] <mdiggory> Usage event config would be added here instead
[21:20] <mdiggory> https://github.com/DSpace/DSpace/tree/master/dspace/config/spring
[21:20] <kompewter> [ DSpace/dspace/config/spring at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/tree/master/dspace/config/spring
[21:21] <PeterDietzhome_> i.e. not webapp/web-inf/applicationContext https://github.com/osulibraries/DSpaceOSUKB/blob/rebase-implement-elasticsearch/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/WEB-INF/spring/applicationContext.xml#L40
[21:21] <kompewter> [ DSpaceOSUKB/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/WEB-INF/spring/applicationContext.xml at rebase-implement-elasticsearch · osulibraries/DSpaceOSUKB · GitHub ] - https://github.com/osulibraries/DSpaceOSUKB/blob/rebase-implement-elasticsearch/dspace-xmlui/dspace-xmlui-webapp/src/main/webapp/WEB-INF/spring/applicationContext.xml#L40
[21:21] <mdiggory> more likely... here
[21:21] <mdiggory> https://github.com/DSpace/DSpace/tree/master/dspace/config/spring/api
[21:21] <kompewter> [ DSpace/dspace/config/spring/api at master · DSpace/DSpace · GitHub ] - https://github.com/DSpace/DSpace/tree/master/dspace/config/spring/api
[21:21] <mdiggory> we won't need the interted injection we are doing any longer.
[21:21] <mdiggory> interted = inverted
[21:24] <mdiggory> beans would just be
[21:24] <mdiggory> <!-- Inject the Default LoggerUsageEventListener into the EventService -->
[21:24] <mdiggory> <bean class="org.dspace.usage.LoggerUsageEventListener"/>
[21:24] <mdiggory>
[21:24] <mdiggory> <!-- Inject the Default LoggerUsageEventListener into the EventService -->
[21:24] <mdiggory> <bean class="org.dspace.statistics.SolrLoggerUsageEventListener"/>
[21:24] <mdiggory> autowiring would be responsible for injection
[21:25] <mdiggory> finally default injections would probably be in dspace-api and dspace-common-api
[21:26] <PeterDietzhome_> so.. instead of wiring my usage-event-listener in dspace-xmlui, its just in the dspace/config/spring/api
[21:27] <PeterDietzhome_> which allows me to not have to edit jspui's applicationContext to ensure that one sends usage-events too
[21:28] * ryscher_ (ae61f77a@gateway/web/freenode/ip. Quit (Ping timeout: 245 seconds)
[21:30] <PeterDietzhome_> that sound good to me, I dont know if i fully understand all of that, but it doesn't sound like a bad path
[21:31] <PeterDietzhome_> i gotta head out now
[21:45] * hpottinger (~hpottinge@mu-162198.dhcp.missouri.edu) has left #duraspace
[21:46] <mdiggory> hey sorry... distracted
[21:46] <mdiggory> yes.. PeterDietzhome_ thats correct
[23:08] * eddies (~eddies@ has joined #duraspace
[23:08] * eddies (~eddies@ Quit (Changing host)
[23:08] * eddies (~eddies@unaffiliated/eddies) has joined #duraspace
[23:11] * mdiggory (~mdiggory@rrcs-74-87-47-114.west.biz.rr.com) Quit (Quit: mdiggory)

These logs were automatically created by DuraLogBot on irc.freenode.net using the Java IRC LogBot.