The DSpace Developers discussed this issue again along with DCAT's comments on Feb 16, 2011.
The general consensus seems to be the following:
- This is a complex task, and we may need to break this up into several steps.
- May be possible to split up into: better metadata validation and improved form management (currently both of these are lumped together into input-forms.xml config file)
- As you may be able to tell from discussion below, the developers themselves are unsure of the best way to approach this. We may need to find a few volunteers to investigate options, especially investigate if there are any third-party, open source tools/form builders which may make this easier (rather than building something new ourselves)
- There is a definite interest amongst the developers to see this happen. But, it may take an institution/developer (or two) volunteering some time for investigation of options and better determining requirements, etc.
Full discussion thread follows:
[20:27] <tdonohue> DCAT does all their reviews on the Wiki, in a new "forum" area: https://wiki.duraspace.org/display/dsforum/Home
[20:27] <tdonohue> (can everyone get to that page? It just prompted me for a login...which is odd)
[20:27] <kshepherd> i got to it ok
[20:27] <kshepherd> but i was already logged in..
[20:27] <robint> I cant login
[20:28] <kshepherd> "You must be a registered wiki user to participate."
[20:28] <helix84> same as kshepherd here
[20:28] <stuartlewis> I'm in (as a logged in user)
[20:28] <mhwood> I got it without login
[20:28] <tdonohue> Ok, well, as long as we can all get to it. I know we are supposed to have access to it – i.e. anyone in the community should be allowed to get to it and join discussion
DS-164 has reminded me of another idea i had, but i'll save it for later
[20:29] <tdonohue> So, in any case, I wanted to (1) make everyone aware of this. DCAT is voting on whether they feel these older feature requests still have merit or not. (2), I thought we could all specifically revisit
DS-164 (the first one they reviewed)
[20:29] <mhwood> Wait, wrong link. Got in with login.
[20:29] <tdonohue> https://wiki.duraspace.org/pages/viewpage.action?pageId=23268096&focusedCommentId=25067837#comment-25067837
[20:29] <mdiggory> Just found it earlier and have been at play in there all morning... thanks
[20:30] <grahamtriggs> yeah, we kinda noticed
[20:31] <tdonohue> So, does anyone else have comments / suggestions / ideas on
DS-164? Anyone aware of the status of any work that could eventually meet these needs, or is there more info we want to ask DCAT for (i.e. better requirements, etc?)
[20:31] <tdonohue> (i'm reading mdiggory's input now)
[20:33] <tdonohue> I should also mention that I agree that this would make a great feature. If anyone is interested in tackling it (or even starting to investigate what it would take to tackle it), I'd do my best to support you (as would DCAT).
[20:33] <grahamtriggs> in my case, I'm the one dealing with the .xml files, so it insulates admins from that layer - but we're seeing issues in terms of what people want simply not being capable with the input-forms (or even the proposed DB driven stuff)
[20:33] <mdiggory> In its simples form... smells like Content Models
[20:33] <mdiggory> grahamtriggs: hear hear
[20:33] <grahamtriggs> and that is in the forms needing to be 'designed' more - not just be a collection of fields
[20:34] <mdiggory> again... hear hear... and TBH, the authority control lookup is still not what our clients want
[20:34] <mhwood> Are there existing forms packages we could use?
[20:34] <tdonohue> grahamtriggs – makes perfect sense. have you or anyone at Open Repositories begun to brainstorm this out at all? Or is the dependency on DB driven configs what is really in the way?
[20:36] <grahamtriggs> I've been forging ahead with Freemarker UI - getting browse / search / item viewing in place - haven't quite got to admin or submission - yet... but I was toying with making it possible to just define the fields directly in Freemarker templates
[20:36] <mdiggory> I worry the DB driven drive detail is a red herring... Though it would be nice to put the capability into the Admins hands, theres still some problems with how this is all hodge podge wired together like a pachinka machine
[20:36] * kshepherd would be more inclined to spend dev time/effort on SWORD, EasyDeposit, etc. than submission in jspui/xmlui
[20:37] <mdiggory> kshepherd: we have need for both... and the validation part I write about plays into the SWORD work as well
[20:37] <grahamtriggs> I tend to agree with mdiggory to separate the validation of fields away from the configuration of the forms. The issue of items being 'correct' is not tied just to the submission forms (although the submission forms do need to be able to present useful feedback in the case of error)
[20:37] <kshepherd> i can see that validation is still needed, yeah
[20:37] <tdonohue> kshepherd – we do need people on both sides of things. But, it's perfect fine to prefer one over the other
[20:38] <mdiggory> tdonohue: how diplomatic...
[20:38] <tdonohue> haha
[20:39] <mdiggory> So... I ponder if we can come up with a set of changes to something like the MetadataRegistry that might get us part of the way there
[20:39] <mhwood> Yes, validation belongs to the abstract metadata field, not the box or element or whatever in which it is presented to the machine.
[20:39] <tdonohue> Ok. So, what are the next steps in getting towards something like
DS-164? Trying to tease out the dependencies here. Sounds like we've already hit on a few of the bigger issues...
[20:40] <mhwood> Tag the metadata field with a validator class.
[20:40] <tdonohue> mdiggory – putting validation details in MetadataRegistry?
[20:40] <kshepherd> if we are redesigning the forms, i wonder if it's worth thinking about 'edit item' as well – i've thought of trying to get XMLUI's item edit page using input-forms.xml or similar so that we can use better form elements, auth control lookups etc in edit pages
[20:40] <tdonohue> kshepherd ++ I agree completely
[20:40] <kshepherd> (and potentially keep the old plain textarea forms behind 'Advanced Edit' or something)
[20:40] <tdonohue> (both edit and submit should be using the same mechanism)
[20:41] <mdiggory> ContentModelRegistry... A list of Content Models that have Metadata Fields Assigned to them and some validation rules like (required, optional) and a value syntax rules
[20:42] <mdiggory> we use the submission forms again in the EditMetadata stage in the XMLUI, seems plausable it could be used in the EditItem as well.
[20:42] <tdonohue> hmm...interesting idea. Not sure I'd call them "Content Models". worried that's a loaded term (e.g. Fedora, Hydra, etc. all use that term)
[20:42] <mdiggory> It is purposefully "loaded"
[20:43] <mdiggory> "Schema"?
[20:43] <kshepherd> or we could try and tie it in with templates
[20:43] <mdiggory> I promise not to say... Ontol...
[20:44] <kshepherd> then along with validation rules, you can supply default values, bitstreams etc. as we do with templates currently
[20:44] <grahamtriggs> mdiggory: I was about to say something similar - I wouldn't mind seeing 'input control/data type' and validation tied against the metadata registry. Form design would then place which fields you are interested where you want them, with minor option tweaking (ie. is it repeatable)
[20:44] <tdonohue> this does bring me back to the question that mhwood posed (no one responded): "are their existing forms packages we can use?" Similarly, is there existing validation/schema/content models frameworks we could use?
[20:44] <kshepherd> (the term "templates", i mean, not the actual code)
[20:45] <grahamtriggs> If you are using the same metadata field for different formats of data in different circumstances (ie. item types or different collections), you probably should rethink your field usage
[20:45] <mdiggory> kshepherd: Templates is interesting, but the problem is that we only get so far with the Item model before we need more
[20:45] <mdiggory> I do agree that Template might be replaceable by one or more "item Schema"
[20:45] <mdiggory> especially ina Type driven submission system
[20:47] <kshepherd> here's something that's kinda fun to play with – an html5 forms builder: http://www.reformedapp.com/#home
[20:47] <mdiggory> grahamtriggs: I agree on the last statement, we get allot of requests to change the Label for dc.foo.bar on collection X but not Y
[20:48] <mdiggory> And we then immediately respond with why thats not so wise to do...
[20:48] <grahamtriggs> different metadata = different fields... If you need to expose it in a common field for OAI-PMH, etc., then that's what crosswalks are for
[20:48] <kshepherd> indeed
[20:49] <tdonohue> So, is
DS-162 of definite interest to anyone or their institution? Curious if we have a person or two who is already interested in digging in deeper on some possible options? (this doesn't mean you'd have to "lead" or build it, just that you'd dig around more and report back on some options/issues)
[20:49] <mdiggory> robint: I'm curious about your view on the topic give the work you've done on Type Driven Submission?
[20:50] <tdonohue> err..I meant
DS-164, not DS-162 (obviously)
[20:50] <robint> mdiggory: Slow thinker.
[20:51] <robint> Can see the need for different labels for the same metadata filed though - dc.date
[20:51] <robint> s/filed/field
[20:52] <robint> Claudia has well developed thoughts on this subject
[20:53] <tdonohue> fyi – if any of you have more thoughts on this later as well, please do feel free to comment directly on DCAT's page, or the
DS-162 issue itself. DCAT is watching both of those, and it'd be good to keep discussions moving forward
[20:53] <mdiggory> I tend to agree that if you want to change the meaning of a field by changing its Label. It is better to change the field or at least qualify it
[20:53] <grahamtriggs> If you need different labels, then the data is representing different things. If the data is different, then the field should be. You can always crosswalk all of them into (eg. dc.date) when you expose the fields if you need to
[20:54] <robint> Are our DC qualifiers not just labels of a sort ?
[20:55] <mdiggory> Old Search/Browse adapted to this through merging metadata fields into single search or browse fields, SOlr currently doesn't "
[20:55] <mdiggory> merge"
[20:55] <kshepherd> yep. my stuff is only ever proper OAI-DC or MODS or whatever after crosswalk and serialisation... before that it's all sorts of crazy schemas and fields
[20:55] <kshepherd> mdiggory: very easy to do in schema though
[20:55] <kshepherd> i have an example as a blog post
[20:55] <grahamtriggs> Granted, there are some curiosities with Dublin Core - ie. titles and book chapters, etc. but that's just crappy, ambiguous dublin core. Store it internally in something that isn't DC and isn't ambiguous, and crosswalk it to DC on the way out
[20:55] <mdiggory> kshepherd: to a degree
[20:56] <mdiggory> but yes, good catch there, and we need some more of that kind of documentation
[20:56] <kshepherd> yeah i have some more to write, too
[20:57] <mhwood> Is the problem separable into nicely-sized subproblems? F'rinstance, can we move validation information away from the forms separately from redoing the presentation? Are there other subproblems?
[20:58] * tdonohue is reminded that many of these discussions seem to circle back to improving Metadata. We need to schedule a special topics meeting on improving how we manage Metadata, etc
[20:58] <kshepherd> just on a tangent, i notice one of the DCAT requests is
[20:58] <mdiggory> Also back to the forms issue, if the validation rules are abstracted from the form definition, it really opens the doors to experiment with new form technologies. Possibly even allot more JSON / AJAX driven Forms handling
[20:59] <kshepherd> and i agree with the JIRA comments.. this should be a curation task
[20:59] <tdonohue> kshepherd – yes,
DS-638 is another they are interested in
[21:00] <tdonohue> kshepherd – I'd encourage you & everyone else to comment on DCAT recommendations outside of this meeting as well. So, don't hesitate to add thoughts/suggestions or ask questions of DCAT. I'm trying to get better about that myself
[21:00] <mdiggory> http://www.reformedapp.com/ ... eh.. whats so easy about filling out lots of individual forms to generate a form?
[21:02] <tdonohue> Ok. it sounds like discussion of
DS-164 is slowing down a bit. I'll post a summary to DCATs page. Please feel free to add more comments there, etc. If you come across and idea or something that could be worth investigating more, let us know!
[21:02] <mhwood> Do we have a good grasp of what sites want from the forms presentation layer? Other than "not so much icky XML".
[21:02] <mdiggory> Per
DS-638 the BitstreamFormatRenovation would have introduced Pronom/Droid in this space...
[21:02] <tdonohue> mhwood – that's something we can ask DCAT for, better requirements. Right now, I think the only request, is 'not XML based, please'